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1.0  INTRODUCTION 


1 . 1  Background 

One  of  the  most  complex  problems  facing  Army  planners  Is  the 
design  and  development  of  effective  training  systems,  particularly  where 
the  use  of  operational  equipment  for  training  purposes  Is  Impractical. 

A  number  of  situations  necessitate  constraint  In  the  use  of  operational 
equipment  In  a  training  role.  These  Include  reduced  military  budgets 
with  consequent  reduced  availability  of  actual  hardware  for  training 
purposes,  reduced  availability  of  large-scale  training  areas  and  ranges, 
and  finally,  growing  concern  for  the  ecological  damage  which  can  arise 
from  mechanlied  field  engagements. 

In  order  to  deal  with  the  limitations  resulting  from  the  reduced 
use  of  operational  hardware  the  Army  has  turned  Increasingly  to  the  use 
of  training  devices  which  simulate  the  operational  situation.  These 
training  devices  are  designed  to  meet  the  needs  of  a  variety  of  students 
who  enter  the  training  situation  lacking  varying  degrees  of  knowledge  or 
skill.  To  the  extent  that  exposure  to  the  training  devlce(s)  Imparts 
necessary  knowledge  and  facilitates  performance  at  specified  criterion 
levels  on  an  operational  task,  the  training  device  Is  judged  to  be  ef¬ 
fective. 

A  basic  problem  Is  how  to  plan  for,  design,  and  develop  a  training 
system  Incorporating  the  different  kinds  of  devices  which  will  prove  to 
be  effective  for  a  particular  training  and  operational  situation.  The 
development  starts  with  a  statement  of  the  requirement  for  training  (e.g., 
a  new  weapon  system  Is  being  developed  for  which  trained  operators  will 
eventually  be  required;  or,  a  training  need  (s  Identified  which  cuts 
across  several  weapons  systems).  The  next  step  Is  to  Identify  what  needs 
to  be  trained  (the  training  deficits  possessed  by  students  who  will  be 
exposed  to  the  training)  and  to  specify  the  general  training  system  which 
will  meet  these  objectives.  At  this  level  the  specification  Is  still  a 
"functional"  one,  oriented  toward  goals  and  objectives  as  opposed  to 
hardware. 


A S'  planning  of  the  training  program  progresses,  the  level  of  detail 
also  Increases  and  the  "how"  of  training  begins  to  be  explored.  Decisions 
are  made  regarding  classroom  versos  on-the-job  training,  the  length  of 
the  course(s),  and  the  requirements  for  training  devices  and  aids  to  sup¬ 
port  training.  Many  of  these  decisions  are  fairly  straightforward  and 
can  be  made  by  experienced  training  specialists  familiar  with  the  personnel 
needs  and  resources  of  the  Army.  Questions  regarding  training  devices, 
however,  are  not  readily  amenable  to  such  a  pr>c*n.  How  and  when  to  use 
them,  how  to  design  them,  and  what  to  spend  on  tnem  are  Issues  which.  In 
the  past,  have  been  dealt  with  In  a  fairly  arbitrary  manner  due  to  the 
lack  of  objective  bases  on  which  to  make  such  decisions.  Sound  Instruc¬ 
tional  decisions  regarding  the  use  of  training  devices  are  contingent  upon 
the  development  of  a  conceptual  framework  and  methodology  which  can  be 
employed  to  forecast  training  device  effectiveness. 

The  primary  goal  of  the  present  project  Is  the  development  and 
evaluation  of  a  model  which  can  be  used  to  predict  and  to  evaluate  the 
effectiveness  of  training  devices.  The  modeling  Is  particularly  aimed  at 
describing  how  device  design,  device  use,  training  strategy,  and  individual 
ability  interact  to  Influence  device  effectiveness.  Standards  of  ef¬ 
fectiveness  Include  both  acquisition  and  transfer  of  military  skills  with 
emphasis  placed  upon  transfer  from  training  to  operational  settings.  As 
described  In  the  preceding  reports  on  this  project  (Wheaton,  Rose, 
Fingerman,  Korotkln,  and  Holding,  1974;  1976),  the  first  step  In  develop¬ 
ing  the  model  was  to  examine,  and  when  possible,  to  build  upon  previous 
efforts.  Toward  this  end,  past  methods  and  models  dealing  with  the 
design  or  evaluation  of  training  devices  were  reviewed,  general  theories 
of  transfer  were  studied,  and  a  host  of  specific  variables  were  surveyed, 
particularly  in  terms  of  their  Impact  on  transfer.  Based  upon  a  synthesis 
of  these  Inputs  a  preliminary  model  was  formulated  for  predicting  the  ef¬ 
fectiveness  of  a  given  training  device  at  the  various  stages  In  Its 
development.  In  the  training-content  by  training-process  model  which 
emerged,  device  effectiveness  was  viewed  as  a  function  of;  (1)  the 
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potential  for  transfer,  (2)  the  magnitude  of  the  trainees'  learning 
deficit,  and  (3)  the  appropriateness  of  the  training  techniques  used  to 
overcome  that  deficit.  Subsequent  efforts  have  sought  to  refine  and 
clarify  the  model  and  to  develop  a  plan  for  field  testing  and  evaluation. 

1 . 2  Purpose  of  the  Report  j 

-As  the  second  document  tn  the  series,  $|>e  present  report  Is 
concerned  with  the  development  of  an  updated  form  of  the  transfer  of 
training  model,  and  the  synthesis  of  Its  components  Into  a  standardized 
method  for  Its  application.  Major  concerns  are  determination  of  the  kinds 
of  data  necessary  to  apply  the  model,  an  assessment  of  the  availability  of 
the  needed  Information,  and  the  development  of  a  feasible  and  reliable 
set  of  procedures  for  processing  the  data  to  generate  predictions  of 
potential  training-device  effectiveness.} 

The  following  sections  of  the  report  outline  the  steps  followed  to 
achieve  these  objectives. ’^Section  2.0  outlines  the  rationale  and 
development  of  the  predictive  model  Into  Its  current  form.  In  generating 
predictions  about  the  training  effectiveness  of  a  given  training  device, 
the  model  combines  data  concerning  the  device's  transfer  potential,  the 
learning  deficits  of  the  trainee  population,  and  the  extent  to  which  the 
device  incorporates  various  training  principles  and  techniques  which  have 

been  shown  to  have  potential  for  enhancing  training  effectiveness. 

;  • 

‘l  Section  3.0  of  the  report  describes  the  detailed  procedures  that 
are  necessary  to  generate  predictions  from  the  model.  Procedural  issues 
include:  the  data  which  are  needed,  as  well  as  how  they  are  to  be 
acquired  and  processed ,/\The  first  procedural  step  Involves  the  determina¬ 
tion  of  transfer  potential  for  the  device  In  question.  The  model  presents 
transfer  potential  as  a  joint  function  of:  (1)  task  coranunallty  between 
the  training  device  and  the  operational  task,  and  (2)  similarity  between 
the  device  and  the  operational  situation.  The  second  step  Involves  the 
determination  of  a  Learning  Deficit  for  the  trainees  who  will  be  trained 
on  the  device.  The  third  step  requires  an  assessment  of  the  extent  to 
which  the  various  training  steps  In  the  device  utilize  various  principles 
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of  training.  The  final  procedural  step  In  applying  the  model  involves 
the  process  by  which  the  data  from  the  other  steps  are  combined  Into  an 
index  or  indices  which  reflect  the  device's  effectiveness.  The  Indicators 
relate  to  the  percentage  of  training  time  saved  by  using  the  device  as 
well  as  to  various  measures  of  trainee  proficiency  after  a  given  amount 
of  exposure  to  the  device.  The  model  also  allows  for  an  assessment  of 
transfer  that  takes  Into  account  the  type  and  amount  of  supporting 
classroom  Instruction  which  the  trainees  experience  before  practice  on 
the  device. 

Section  3.0  also  describes  in  detail  the  step-by-step  procedures 
that  the  project  staff  followed  In  performing  an  experimental  appli¬ 
cation  of  the  model.  Two  training  tasks  used  for  this  purpose  were: 

(1)  fire  main  gun  (H60A  tank)  using  the  M-32  sight,  and  (?)  apply 
burst-on-target  adjustment  of  fire  using  the  M-32  sight.  Tour  devices 
were  processed.  These  were  the  17-4  Burst-on-Target  Trainer,  17-B4 
Burst-on-Target  Trainer,  M-55  Conduct-of-FIre  Trainer,  and  SIMFIRE. 

These  devices  were  selected  In  a  manner  designed  to  yield  at  least 
modest  variation  In  their  predicted  efficacy  for  training  the  two 
criterion  tasks  so  that  the  model's  utility  could  be  assessed  across  a 
range  of  predictions. 

The  purpose  of  this  preliminary  application  was  twofold.  First,  It 
was  designed  to  assess  the  feasibility  of  applying  the  model .  Feasibility 
Includes  an  assessment  of  the  time  In  a  training  device's  "life  cycle"  at. 
which  the  model  can  be  applied,  either  at  the  Training  Device  Requirement 
(TOR)  stage,  or  at  the  prototypk  device  stage.  Another  aspect  of 
feasibility  concerns  the  potential  application  of  the  model  for  systems 
versus  non-systems  training  devices.  The  second  purpose  of  the  applica¬ 
tion  was  to  determine  the  reliability  of  the  procedures.  In  most  cases, 
modeling  data  were  collected  Independently  from  four  senior  project  staff 
members  and  reliability  Indices  were  computed.  These  are  reported  In 
Section  3.0.  This  section  also  discusses  the  problems  that  the  project 
staff  encountered  In  applying  the  model  and  the  resolutions  which  were 
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achieved.  This  Information  should  prove  Invaluable  In  future  applica¬ 
tions  and  as  guidance  for  Army  personnel  who  may  wish  to  apply  the 
model . 

Section  4.0  presents  the  development  and  derivation  of  a  predictive 
equation  using  as  parameters  the  outputs  of  the  analyses  discussed  In 
Section  3.0.  Section  5.0  consists  of  a  discussion  of  the  results,  and 
suggestions  for  further  research  In  the  area. 
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2.0  THE  MODEL 


2 . 1  tid  e  ground 

The  first  report  in  this  series  (Wheaton,  et  a).,  1976)  presented 
a  preliminary  structural  model  for  use  in  forecasting  training  device 
effectiveness.  This  preliminary  model  Incorporated  most  of  the  central 
issues  Involved  In  training  device  effectiveness  that  were  revealed 
through  an  analysis  of  previous  models,  methods,  and  empirical  data. 

The  model  dealt  with  t*»  major  classes  of  variables: 

1.  Those  associated  with  developing  a  training  device  which  does, 
in  fact,  elicit  the  behaviors  which  are  required  in  the  opera¬ 
tional  situation;  these  wete  termed  "Appropriateness"  variables. 

?.  Those  variables  associated  with  actually  learning  these 
behaviors;  these  were  called  "Lfficiency"  variables. 

Under  "Appropriateness,"  the  central  issue  was  the  transfer 
potential  of  the  device.  Assuming  that  the  trainee  became  proficient  on 
the  tasks  presented  in  the  training  situation,  would  he  then  meet  the 
training  requirements?  To  deal  with  this  question,  three  types  of 
analyses  were  proposed:  (1)  a  communal  1 ty  analysis,  (2)  a  criticality 
analysis,  and  (3)  a  similarity  analysis. 

In  addressing  the  "Lfficiency"  issues,  two  major  analyses  wore  pro¬ 
posed.  The  first  Involved  a  detern.inat  ion  of  the  trainee's  learning 
deficit:  an  assessment  of  what  trainees  were  actually  required  to 
learn.  This  was  addressed  by  three  proposed  analyses:  (1)  determining 
whether  appropriate  skills  and  knowledge  were  already  In  the  trainee's 
repertory;  (2)  establishing  the  proficiency  requirements  for  the 
criterion  transfer  task;  and  (3)  estimating  how  difficult  it  would  be 
to  learn  the  task.  The  second  major  analysis  subsumed  under  "Efficiency" 
was  the  Training  Techniques  and  Principles  analysis.  This  proposed 
analysis  was  an  attempt  to  make  direct  use  of  empirical  data  and  training 
principles  for  a  specific  situation  in  order  to  depict  the  efficacy  of 
training. 
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The  basic  Input  data  for  all  of  these  proposed  analyses  was 
presumed  to  stem  directly  from  or  to  be  derivable  from  task  analyses  of 
the  training  and  the  operational  situations.  The  prel 1ml nary  model  pro¬ 
posed  two  complementary  schemes  for  conducting  the  task  analyses,  thereby 
determining  the  "Training  Content":  (1)  a  detailed  task  description,  and 
(2)  a  behavioral  taxonomic  classification. 

In  summary,  the  preliminary  model  was  represented  as  a  training- 
content  by  training-process  matrix.  The  content  axis  consisted  of  task 
analytic  data,  while  the  process  axis  was  made  up  of  two  major  headings, 
Appropriateness  and  Efficiency,  and  several  subheadings.  A  functional 
model  was  only  implied  In  the  previous  report;  basically,  It  was  assumed 
that  the  Inputs  to  the  Appropriateness  and  Efficiency  analyses  would  be 
task  or  subtask  descriptions  (and  the  behavioral  categories  for  these 
tasks)  of  the  operational  system  and  the  training  situation,  combined 
with  a  physical  description  of  the  operational  and  training  equipment. 

The  precise  nature  of  the  measurements  to  be  taken,  the  resultants  of 
those  Individual  analyses,  and  how  these  measures  would  be  combined  were 
unspecified  In  the  prel 1ml nary  model . 

2 •  ?  Current  Structural  and  Functional  Model 

With  minor  exceptions,  the  model  in  Its  current  state  of  development 
retains  the  basic  structure  of  the  preliminary  model.  While  specific 
decisions  regarding  the  implementation  of  the  various  analyses  have  been 
made,  the  basic  rationale  for  the  general  types  of  analyses  to  be  per¬ 
formed  has  remained  unchanged.  Training  device  effectiveness  Is  still 
viewed  as  a  function  of  the  transfer  potential  of  the  device,  the 
learning  deficit  of  the  trainers,  and  the  extent  to  which  appropriate 
training  techniques  are  utilized  In  the  device.  As  mentioned  in  the 
previous  report,  training  effectiveness  In  general  will  be  moderated  by 
a  host  of  potent  variables  external  to  the  device  Itself,  such  as  device 
acceptance,  ether  Instructional  support,  etc.  While  It  Is  still  felt 
that  many  of  these  variables  would  more  appropriately  be  considered  In  a 
training  system  effectiveness  model,  provision  has  been  made  for  an 
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"extended"  application  of  the  current  device  effectiveness  model .  For 
instance,  it  has  been  found  that  the  type  and  amount  of  supporting 
classroom  instruction  can  be  incorporated  Into  the  learning-deficit 
portion  of  the  model.  Other  examples  of  this  extended  model  application 
will  be  presented  below, 

Figure  1  presents  the  structural  and  functional  model  at.  its 
present  stage  of  development,  The  structural  model  is  divided  under 
three  major  headings:  Inputs,  Processes,  and  Outputs.  Functional 
relationships  are  Indicated  by  arrows  leading  from  Inputs  through 
processes  to  outputs.  In  order  to  minimire  perceptual  confusion, 
several  arrows  have  been  omitted  from  the  figure;  these  omissions  will 
be  discussed  below,  along  with  their  appropriate  locations  and  functions, 
The  following  discussion  of  the  model  Is  organized  around  the  three 
major  analyses  comprising  the  processes  by  which  Input  data  are  trans¬ 
formed  Into  outputs.  These  major  analyses  are:  Transfer  Potential, 
Learning  Deficit,  and  Training  Techniques,  Detailed  discussion  of  the 
procedures  Involved  in  conducting  eat h  analysis  will  be  presented  in 
Section  ,j ,  D  below. 

P.L’.l  Transfer  Potential.  The  model  presents  transfer  potential  as 
a  joint  function  of  (1)  task  commonality  between  the  training  situation 
and  the  operational  setting,  and  (.■'}  similarity  between  the  training 
device  and  the  operational  equipment.  Task  c  onmiun  a 1 1 ty  refers  to  whether 
a  specific  task  in  the  operational  situation  is  represented  In  the  train¬ 
ing  device.  Conrounal 1 ty  can  be  assessed  in  two  ways,  first,  coi mural 1 ty 
tan  be  determined  from  the  actual  way  that  the  devire  Is  currently  used 
by  the  Army.  For  example,  for  the  subtask,  "Indexes  anrnunltlon  into  the 
computer,"  SIMriRL  has  no  connuna 1 1 ty  with  the  operational  task  since 
trainees  are  presumed  not  to  index  anmunltton  in  the  SIMFIRE  trainer. 

The  other  wa.y  of  viewing  communal  Ity  is  to  assess  "potential"  corammallty 
between  the  device  and  the  operational  situation,  In  the  example  above, 
SIMFIRr.  would  have  potential  communallty  with  the  operational  task, 
since  it  would  certainly  be  possible  for  trainees  to  Index  ammunition  In 
the  SIMRIRL  trainer. 
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Figure  1.  Structural  and  functional  ■ode!  of 
training  device  effectiveness. 


Similarity  refers  to  (1)  the  physical  similarity  between  the  displays 
and  controls  in  the  training  device  and  those  in  the  operational  situa¬ 
tion,  and  (?)  the  degree  of  functional  similarity  between  the  controls 
and  displays  in  the  two  situations.  Physical  similarity  is  what  has 
traditionally  been  referred  to  as  “fidelity."  It  refers  to  the  appearance 
and  physical  characteristics  of  the  displays  and  controls  In  the  training 
device  and  the  operational  situation.  Functional  similarity  is  a  newer  con¬ 
cept  which  refers  to  what  the  trainee  actually  does  to  the  controls  and  dis¬ 
plays  In  the  two  situations.  The  concept  is  based  on  Information-processing 
theory,  and  utilizes  the  Idea  of  Information  transmission  (Fitts  and  Posner, 
1967).  Two  basic  decisions  are  made  In  determining  functional  similarity, 
First,  are  the  informat  Ion-process  1 ng  functions  ot  the  displays  and  controls 
in  the  operational  situation  represented  realistically  in  the  training  device? 
(unction  refers  to  such  formal  Informat  ion- processing  activities  as  conser¬ 
vation,  reduction,  or  generation.  For  example,  a  task  involving  conservation 
is  one  in  which  the  output  has  some  fixed  relationship  to  stimulus  events, 
so  that  the  stimulus  (input)  can  be  Inferred  precisely  from  the  response 
(output).  To  Illustrate,  in  the  operational  situation,  the  earphones  serve 
the  function  of  consorvat ion :  they  transmit  signals  from  thr  tank  contnander 
to  the  gunner  regarding  type  of  ammunition,  etc.  In  the  training  situation, 
the  earphones  are  not  present;  however,  the  same  information  conservation 
function  is  maintained  since  the  same  information  is  still  transmitted  to 
the  gunnery  trainee.  The  second  decision  is  with  respect  to  the  amount  of 
Information  transmitted  in  the  two  situations,  for  example,  in  the  operational 
Situation  assume  that  the  ammunition  command  reduces  the  gunner's  uncertainty 
as  to  which  of  six  types  of  rounds  he  will  have  to  Index.  If,  In  a  training 
device,  it  Is  possible  to  Index  only  two  different  rounds,  the  amount  of 
Information  In  the  two  situations  Is  different.  The  concept  of  functional 
similarity  will  be  further  elaborated  In  the  procedures  section  (3.0)  below. 

Inputs  to  the  Conmunallty  analysis  consist  of  subtask  descriptions 
for  the  device  and  the  operational  situation,  The  Physical  Similarity 
analysis  requires  a  list  and  description  of  the  controls  and  displays  for 
the  device  and  the  associated  operational  equipment,  and, the  Func¬ 
tional  Similarity  analysis  requires  a  list  of  control  and  display 
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functions  In  the  two  situations.  Operational  definitions  for  these 
Inputs,  along  with  the  procedures  for  obtaining  the  Information,  are 
presented  In  Section  3.0.  The  outputs  from  each  of  these  analyses  are 
combined  to  arrive  at  a  Subtask  Transfer  Potential  value  for  each  of  the 
subtasks.  Again,  combinatorial  rules  will  be  presented  In  later  sections 
of  this  report, 

The  preliminary  model,  in  addition  to  proposing  a  Cormunallty  and 
Similarity  analysis  for  Transfer  Potential,  also  Included  a  Criticality 
analysis.  The  basic  idea  was  to  determine  which  of  the  tasks  Identified 
by  the  training  objective  and  found  in  the  operational  situation  would  be 
most  critical  to  have  represented  in  the  training  device.  It  became  ap¬ 
parent,  howevor,  that  no  task  was  a  priori  more  critical  than  any  other 
for  operational  performance;  In  a  sense,  all  subtasks  are,  by  definition, 
equally  important  since  failure  to  perform  any  one  of  them  might  result 
In  failure  to  perform  the  entire  task  adequately.  Criticality,  therefor*, 
Is  currently  viewed  from  the  perspective  of  the  trainee:  what  tasks  or 
subtasks  require  the  most  training?  Thus,  criticality  has  been  subsumed 
under  the  next  major  analysis,  learning  Deficit. 

2-2.?  learning  Oeflclt.  This  analysis  essentially  Involves 
determining  the  relationship  between  the  level  of  proficiency  that  a 
trainee  has  prior  to  training  and  the  level  of  proficiency  that  Is  re¬ 
quired  In  order  to  be  able  to  perform  the  operational  task  to  criterion 
specifications.  There  are  two  procedures  Involved  In  this  analysis: 

(1)  a  determination  of  the  "objective"  difficulty  of  the  subtasks  in¬ 
volved  In  operational  performance:  and  (2)  an  assessment  of  the  specific 
deficits,  in  terms  of  skill  and  knowledge  requirements,  of  the  trainees. 
The  first  determination  Involves  a  ranking  of  the  relevant  subtasks  by 
difficulty,  independent  of  the  specific  trainee  population.  The  second 
assessment  involves  a  rating,  for  each  skill  and  knowledge  requirement, 
of  the  required  level  of  proficiency  and  the  hypothesized  level  of 
proficiency  of  a  given  trainee  population. 
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Inputs  for  these  analyses  are  again  the  subtask  descriptions  for  the 
operational  tasks,  and  naturally,  the  training  objective.  In  addition, 
a  list  of  the  skills  and  knowledge  required  for  each  of  the  subtasks 
serves  as  input,  This  analysis  is  performed  Independent  of  any 
particular  training  device;  Learning  Deficit  depends  solely  on  a 
specification  of  what  trainees  must  know  at  the  end  of  training.  As 
mentioned  above,  "Criticality"  now  refers  to  those  tasks  which  have  the 
largest  learning  deficits;  these  are  the  tasks  that  are  most  necessary  to 
have  represented  in  any  training  device.  The  output  of  the  Learning 
Deficit  analysis  Is  a  score  representing  the  difficulty  of  a  subtask, 
weighted  by  the  severity  of  the  specific  deficits  on  that  subtask  for  a 
given  trainee  population.  As  suggested  above,  It  Is  possible  to  Incor¬ 
porate  knowledge  about  auxiliary  classroom  instruction  at  this  stage  of 
the  overall  analysis,  for  example,  classroom  Instruction  could  provide 
some  of  the  necessary  knowledge  that  would  otherwise  contribute  to  a 
greater  deficit  rating;  this  Information  would  result  In  a  change  In  the 
weights  assigned  to  a  particular  subtask. 

?.?,3  Training  Techniques.  The  third  major  analysis  Involves  an 
assessment  of  the  extent  to  which  the  various  training  steps  In  the 
device  utilize  various  principles  and  techniques  of  training.  These 
principles  and  techniques  were  generated  from  an  analysis  of  the  existing 
training  and  transfer  literature.  The  basic,  procedure  Is  to  compare  a 
specific:  device  configuration  against  these  training  techniques  and 
determine  whether  or  no*  ^nd  to  what  degree)  the  device  conforms  with  or 
violates  good  practice. 

The  Inputs  to  this  analysis  are  unspecified  in  Figure  1.  In  effect, 
any  and  all  Information  obtained  from  the  preceding  analyses  Is  con¬ 
sidered  as  input;  this  Includes  the  training  objective,  skill  and 
knowledge  requirements,  similarity  ratings,  and  so  forth.  The  principles 
themselves  have  been  rated  with  respect  to  empirical  and  theoretical 
support. 
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logistlcally,  the  Training  Technique  analysis  Is  a  most  time-consuming 
activity;  presently,  there  are  approximately  one  hundred  principles  and 
techniques  to  be  evaluated  for  each  subtask,  for  each  device  under  consid¬ 
eration.  Various  strategies  have  been  employed  to  facilitate  this 
analysis.  One  technique  Is  to  categorize  each  subtask  and  principle  Into 
an  appropriate  behavioral  category  (see  Section  3.6,  and  Appendix  A); 
this  reduces  the  number  of  principles  scanned  for  each  subtask.  Another 
technique  Is  to  divide  the  principles  Into  categories  which  pertain  to 
various  aspects  of  performance.  In  the  examples  processed  for  the 
present  report,  these  categories  were:  stimulus  parameters,  response 
parameters,  and  feedback  parameters.  Further  modification  and  organiza¬ 
tion  of  this  analysis  Is  being  considered. 

The  output  of  this  analysis  Is  a  rating,  for  each  subtask,  of  the 
potential  effectiveness  of  a  particular  device  for  overcoming  a  specified 
learning  deficit  In  a  given  subtask.  This  output  Is  then  combined  with 
the  outputs  from  the  Transfer  Potential  and  Learning  Deficit  analyses  to 
produce  a  subtask  device  effectiveness  rating.  Finally,  the  ratings  for 
each  of  the  subtasks  are  combined  to  produce  an  overall  device  effective¬ 
ness  rating. 

Succeeding  sections  will  further  elaborate  the  derivation  of  Inputs, 
the  specific  methodology  for  the  processes,  and  the  mathematical  and 
practical  rationale  for  the  combinatory  rules.  The  specific  procedures 
have  Implications  for  Informational  requirements  of  the  model  as  well  as 
for  experimental  paradigms  necessary  to  test  It.  Aside  from  the  specific 
application,  however,  we  feel  that  the  model  In  Its  present  form  Incor¬ 
porates  the  various  considerations  applicable  to  the  evaluation  of 
training  device  effectiveness.  While  the  adequacy  of  the  procedures  and 
the  relations  between  the  various  outputs  remain  to  be  empirically 
validated,  the  concepts  and  constructs  underlying  the  model  were  generated 
from  a  thorough  consideration  of  existing  literature,  expert  opinion, 
and  practical  guidelines  of  training.  As  such,  tha  model  can  be  considered 
apart  from  Its  specific  application  as  the  current  best  guess  as  to  what 
should  be  considered  In  the  evaluation  of  a  device. 
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3.0  PROCEDURES  FOR  APPLICATION  OF  THE  MODEL 

.1 . 1  Overview 

In  this  section  procedures  are  presented  for  application  of  the  model 
to  generate  forecasts  about  the  effectiveness  of  training  devices,  The 
intention  is  to  describe  the  analytic  steps  In  sufficient  detail  to  enable 
others  to  apply  the  model.  In  the  Army's  case  the  required  analyses  Mould 
be  performed  by  personnel  experienced  in  task  analysis  and  In  derivation  of 
training  objectives.  In  the  following  portions  of  this  section  (3. 2*3. 6)  five 
bask  analyses  are  described  including:  Task  Comnunal  Ity  analysis  (3.2), 
Physical  Similarity  analysis  (3.3),  Functional  Similarity  analysis  (3.4), 
Learning  Deficit  analysis  (3.5),  and  Training  Technique  assessment  (3.6), 

In  the  final  portion  of  this  section  (3.7),  procedures  are  presented  for 
determining  summary  index  values  for  the  three  components  of  the  structural 
model:  Transfer  Potential,  Learning  Deficit,  and  Training  Techniques. 

Each  analysis  is  described  according  to  a  standard  format.  First, 
the  input  data  requirements  are  Indicated  and  the  steps  necessary  to  generate 
these  inputs  are  discussed.  Next,  the  specific  procedural  steps  Involved 
in  conducting  the  analysis  are  described.  Examples  are  given  of  how 
actual  input  data  were  analyzed  by  the  project  stafl,  The  results 
of  this  application  are  presented  along  with  a  discussion  of  the  feasibility 
of  obtaining  the  required  input  data,  the  reliability  of  the  procedures 
followed,  and  the  problems  encountered  and  their  resolution. 

In  each  Instance  when  a  specific  analysis  was  attempted,  the  same 
test-bed  was  employed.  The  effectiveness  of  four  devices  in  meeting  two 
different  training  objectives  was  determined.  The  training  devices  con¬ 
sisted  of:  (1)  the  17-4  Burst-on-Target  Trainer;  (2)  the  17-B4  Burst- 
on-Target  Trainer;  (3)  the  M55  Conduct  of  Fire  Trainer;  and  (4)  SIMFIRE. 

The  tasks  underlying  the  two  training  objectives  were:  (1)  "fire  the 
main  gun  (M60A1  tank)  using  the  M-32  sight";  and  (2)  "apply  burst-on 
target  adjustment  of  fire  using  the  M-32  sight." 

3 • 2  Task  Communallty  Analysis  (TCA) 

As  located  above  In  5ect1on  2.0,  the  first  major  analytic  step 
in  applying  the  model  Is  to  conduct  a  Task  Communal Ity  analysis.  The 
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basic  purpose  of  this  first  step  Is  to  describe  the  overlap  In  training 
content  which  exists  between  the  operational  situation  and  any  designated 
training  device.  The  amount  of  this  overlap  Is  crucial  since  It  Is  as¬ 
sumed  that  the  potential  for  transfer  of  training  (see  Figure  1)  will 
Increase  as  a  direct  function  of  the  degree  to  which  criterion  task 
relevant  content  Is  contained  within  the  training  device. 

Specifically,  overlap  of  content  refers  to  the  degree  to  which  sub¬ 
tasks  comprising  the  operational  criterion  situation  (l.e.,  fire  main  gun 
using  M-3Z  sight)  are  represented  In  the  device.  On  this  basts  commune Hty 
Is  said  to  exist  when  the  trainee  can  perform  and  practice  a  subtask  In 
the  training  situation  which  Is  also  performed  In  the  operational 
situation. 

3.2.1  Data  Requirements.  The  first  and  most  basic  data  require¬ 
ment  In  conducting  TCA ,  or  any  of  the  other  analyses  for  that  matter,  Is 
a  detailed  statement  of  the  training  objective.  The  Importance  of 
satisfying  this  requirement  cannot  be  emphasized  strongly  enough.  As 
implied  In  Figure  1  above,  It  Is  the  stated  training  objective  which 
serves  to  focus  attention  on  a  specific  criterion  situation  Including: 

1)  the  precise  nature  of  the  task  to  be  learned;  2)  the  conditions  of 
task  performance  during  transfer;  and  3)  the  performance  standard(s)  to 
be  met.  Information  on  each  of  these  aspects  of  the  stated  training 
objective  Is  vital  for  application  of  different  portions  of  the  model. 

The  procedures  for  developing  a  detailed  statenent  of  the  training 
objective  have  been  formalized  and  are  presented  in  CON  REG  350-100-1 
(1972)  and  CON  PAM  350-11  (1973).  As  will  be  discussed  In  a  later  section 
of  the  report,  however,  this  formal  detailing  Is  not  typically  done  dur¬ 
ing  earliest  stages  of  device  development.  Consequently,  the  statements 
of  training  objectives  which  accompany  Training  Device  Requirement  (TOR) 
documents  are  often  frapwntary  and  of  a  very  general  nature.  This  lack 
of  detail  Impacts  upon  application  of  the  model  eerly  In  device  develop¬ 
ment. 
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There  is  a  second  major  data  requirement  for  TCA,  Detailed  task- 
analytic  information  is  needed  regarding  the  operational  criterion  task 
which  has  been  specified  in  the  training  objective;  similar  data  are  needed 
regarding  the  training  task  Itself.  Tor  TCA,  identification  and  listing  of 
the  subtasks  comprising  the  operational  criterion  task  and  the  training 
task  are  essential , 

In  order  to  generate  such  data  the  general  approach  described  in 
CON  PAH  350-11  (1973)  and  the  specific  guidelines  provided  by  Folley 
(1964)  and  Chenzoff  and  Folley  (1965)  may  be  used.  To  provide  Inputs 
for  TCA  the  approach  is  to  break  the  operational  criterion  situation 
down  Into  successively  finer  units  of  description,  stopping  at  what 
constitutes  the  subtask  level  of  detail.  Based  on  Folley's  (1964) 
system  of  description,  a  subtask  may  be  defined  as  an  activity  that  Is 
performed  by  one  person  and  bounded  by  two  events.  An  example  of  a  sub¬ 
task  might  i>r,  "dpon  receipt  of  the  alert  element  of  a  fire  command, 
sets  turret  fxwr  switch  to  the  'ON'  position".  An  event  may  be  defined 
as  a  discrete  and  identifiable  act  or  occurrence.  Examples  would  be, 

"receipt  of  alert  elenent"  and  "switch  in  'ON*  position".  An  activity 
is  defined  as  the  behavior(s)  comprising  a  subtask,  such  as  "setting  a 
switch".  A  task  is  defined  as  a  set  of  two  or  more  subtasks  (e.g.,  "fire 
min  gun  using  the  M-32  sight")  and,  finally,  a  systw.,block  Is  defined 
as  a  set  of  tasksoccurrinfl  at  about  the  same  time  in  system  operation, 
all  d i r ec ted  toward  achieyinjL the  same. sub-objective  in  the  opera ti cm . 

For  the  applications  of  the  model  described  in  the  current  report, 
task  descriptive  data  generated  by  the  U.S.A.  Armor  School,  Fort  Knox, 

Kentucky,  by  bowers  and  Hcf.luskey  (1*76),  and  by  the  project  team  were  subjected 
to  the  detailing  described  above.  From  among  the  65  general  tasks  as¬ 
sociated  with  KOS  1 1 E  (Armor  Crewnan),  one  was  chosen  for  study:  "Fire 
M6Q/M6QA1  Main  Gun".  This  system  block  was  comprised  of  15  tasks,  two 
of  which  were  selected  for  analysis.  As  previously  Indicated,  these 
consisted  of:  1)  "firing  the  main  gun  with  the  M-32  sight";  and  2)  "ap¬ 
plying  the  burst-on-target  (BOT)  method  of  adjusting  fire  using  the  M-32 
sight".  Each  task  was  then  analyzed  In  order  to  generate  a  listing  of 
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Its  component  subtasks.  The  results  of  this  analysis,  shown  in  Table  1, 
served  as  the  primary  input  to  TCA. 

3.2.2  TCA  Procedure.  The  first  step  in  TCA  is  to  construct  a  task- 
cofimunallty  matrix  as  shown  in  Table  2.  Task  and  subtask  Information 
about  the  operational  situation  is  listed  down  the  left  margin.  The 
training  devices  to  be  assessed  are  listed  across  the  top  of  the  page. 
Notice  that  the  listing  of  devices  is  repeated  in  order  to  permit 
separate  analyses  to  be  performed  with  respect  to  potential  and  actual 
conmunality.  Potential  conmuna 1 ity  addresses  the  overlap  between  sub¬ 
tasks  in  the  training  and  operational  settings  which  could  conceivably 
exist,  regardless  of  how  the  device  is  currently  used.  As  such  it 
represents  a  maximum  estimate.  When  data  are  available  describing  how 
the  device  is  actually  used  (i.e.,  which  subtasks  are  practiced)  e 
second  estimate  of  comnunallty  is  possible.  This  second  estimate  will 
be  the  more  accurate  of  the  two,  but  will  usually  be  available  only  In 
cases  where  a  prototype  of  the  device  Is  in  use.  (It  Is  even  conceivable 
that  the  same  device,  evaluated  at  two  different  locations,  could  produce 
different  TCA  estimates,  due  to  differences  In  device  utilization.)  In 
any  event,  comparison  of  the  two  analyses  may  prove  useful.  When  a  large 
discrepancy  exists  between  potential  and  actual  comnunaHty,  It  may 
Indicate  the  desirability  of  revising  current  methods  of  utilization  In 
order  to  take  better  advantage  of  the  device's  potential  for  positive 
transfer. 

The  second  step  In  TCA  consists  of  a  listing  of  the  subtasks  com¬ 
prising  the  training  task.  This  listing  is  accomplished  separately  for 
each  task  and  device  under  consideration.  For  accuracy  and  to  Insure 
reliability  It  Is  suggested  that  this  step  be  carried  out  formally. 
Potentially  valuable  Information  may  be  overlooked  if  one  simply 
considers  each  operational  subtask  and  makes  a  guess  about  its  Inclusion 
In  the  device. 

Armed  with  lists  of  subtasks  for  the  device  and  operational  setting, 
one  can  perform  the  third  and  crucial  step  In  TCA.  for  each  operational 
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TABLE  1 


SUBTASK  LISTING  FOR  MAJOR  TASKS  IN  THE  OPERATIONAL  SITUATION 


TASK:  fire  main  gun  using  primary  sight  (M-32). 

Subtasks: 

1.  Upon  receipt  of  the  alert  element  of  the  fire  command,  places  turret 
power  switch  In  "on"  position.  (Alert) 

2.  Upon  receipt  of  the  ammunition  element  of  the  fire  command,  places 
main  gun  power  switch  In  "on"  position  and  coaxial  machine  gun 
switch  to  the  appropriate  position.  (Select  Gun) 

3.  Index  appropriate  ammunition  Into  the  computer,  using  the  armunltlon 
selector  control.  (Index  Computer) 

4.  Upon  receipt  of  the  target  element  of  the  fire  command,  monitors  unity 
window  for  target,  and,  when  located  announces,  "Identified."  (Monitor 

5.  Upon  receiving  control,  operates  controls  to  place  cross  hairs  of 
sight  on  center  of  target  vulnerability.  (Initial  Aim) 

6.  Tracks  target.  (Track) 

7.  Upon  receipt  of  execution  element  of  fire  command,  checks  final  lay 
of  the  gun  and  applies  appropriate  lead.  (Final  Aim) 

0.  Hears,  "Up,"  announces,  "On  the  way,"  pauses  one  second,  and  squeezes 
trigger.  (Fire) 


TASK:  Adjust  fire  using  Burst  on  Target  (M-32  Sight). 

Subtasks: 

1.  Upon  firing,  monitors  sight  for  target  and  relays  as  necessary  to 
reacquire  the  original  sight  picture.  (Relay) 

2.  Senses  the  round  (observes  burst  In  relation  to  target,  and  determines 
new  aiming  point  on  reticle)  and  begins  to  relay.  (Sense) 

3.  Operates  controls  to  place  new  aiming  point  on  center  of  target 
vulnerability.  (Apply  HOT) 

4.  Tracks  target.  (Track) 

5.  Hears,  "Up,"  announces,  "On  the  way,"  pauses  one  second,  and  squeezes 
trigger.  (Fire) 
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Corrauna!  ity  estimates  ft  or  O'  based  on  a  ccnse-sas  ancr.g  foa r  judges;  interjudge  agreement  ceTculated 
the  independent  estimates  prior  tc  consensus. 


subtask  listed  along  the  left  margin  of  the  task-communal ity  matrix,  the 
analyst  scans  his  list  of  training  subtasks.  If,  in  fact,  a  device  could 
or  does  enable  the  trainee  to  practice  the  subtask  in  question,  a  "1"  is 
entered  in  the  appropriate  cell  under  the  device.  However,  if  that 
particular  subtask  is  not  represented,  a  "0"  is  entered.  This  process 
is  continued  until  all  operational  subtasks  have  been  evaluated.* 

In  making  Judgments  about  comunality  it  is  important  to  remember 
that  the  only  consideration  is  whether  or  not  the  essence  of  the  opera¬ 
tional  task  is  represented;  in  general  the  Judgment  will  be  based  on  the 
activity  or  behavior  comprising  the  subtask,  The  issue  of  how  well  or 
how  faithfully  the  subtask  is  represented  is  dealt  with  during  Similarity 
analysi s, 

3 . y . J  Results.  The  results  of  the  Task  Commonality  analyses  are 
presented  in  Table  2  for  each  of  the  devices  under  examination.  The 
task  communal ity  indices  Indicate  that  communallty  is  quite  high  for  all 
of  the  devices  examined,  and  highest  for  those  devices  (e.g.,  M55  and 
SIMF'IRt.)  which  make  use  of  the  M60A1  tank  itself.  The  17-4  and  17-114 
devices  exhibit  less  communal  1 ty ,  primarily  due  to  the  fact  that  they 
do  not  simulate  moving  targets,  maklnq  it  impossible  to  train  tracking 
(subtasks  1-6,  and  11-4).  It  will  also  be  noted  that  potential  commu- 
na 1 i ty  is  slightly  better  than  actual  cormunality.  This  occurs  for  two 
reasons. 

First,  in  this  application  it  was  assumed  that  the  trainee  does 
not  Index  ammunition  (subtask  1-3)  in  the  two  tank-based  devices. 

Second,  In  actual  use  none  of  the  devices  compels  the  trainee  to  reacquire 
his  original  sight  picture  after  firing  the  main  gun  (subtask  1 1 - 1 ) . 

The  results  of  both  TCA's  are  retained  and  carried  forward  for  use  in 
compiling  summary  Indices. 

*!n  some  cases  there  will  be  additional  subtasks  associated  uniquely 
with  a  device  and  not  found  In  the  operational  setting.  These  sub¬ 
tasks  should  be  footnoted  at  the  bottom  of  the  task-communal  1 ty 
matrix,  and  retained  for  further  analysis. 


3.2.4  Evaluation.  In  general,  TCA  proved  to  be  straightforward, 

there  being  few  problems  In  actually  making  the  ratings.  Rating  ease 

was  undoubtedly  facilitated  by  the  dichotomous  nature  of  the  decision; 
either  the  subtask  was  represented  In  the  training  device  or  It  was 

not. 

It  should  be  emphasized,  however,  that  successful  application  of 
the  TCA  procedure  requires  rather  detailed  task-descriptive  statements 
for  both  the  device  and  the  operational  equipment.  After  careful  review 
of  a  number  of  Training  Device  Requirements  (TOR'S)  provided  by  the  Army, 
the  project  staff  was  of  the  opinion  that  the  level  of  detail  currently 

provided  In  these  documents  would  be  generally  Inadequate  for  a  detailed 

subtask  communallty  analysis.  This  point  Is  underscored  by  the  fact 
that  the  only  difficulties  encountered  In  making  the  ratings  on  existing 
devices  were  due  to  Incomplete  Information  about  the  device  on  the  part 
of  the  staff.  Most  of  this  difficulty  was  experienced  by  staff  members 
who  had  not  actually  seen  the  device  In  question,  These  difficulties 
were  readily  cleared  up  by  other  staff  members  more  familiar  with  the 
device.  Consequently,  TCA  Is  Judged  feasible  given  that  the  required 
task-descriptive  Input  data  can  be  generated. 

In  Table  2  Inter-Judge  agreement  data  are  shown  within  each  major 
task  for  the  four  devices.  Communallty  Judgments  were  obtained  Inde¬ 
pendently  from  four  staff  members  who  had  advanced  training  and  extensive 
experience  In  the  behavioral  sciences.  The  reported  coefficients 
represent  the  proportion  of  cases  In  which  the  Judges  were  In  complete 
agreement.  Thirty-two  judgments  were  obtained  for  each  device  during  the 
analysis  of  Task  I;  during  analysis  of  Task  II,  20  judgments  were  made. 

In  general,  Interjudge  agreement  was  quite  high.  A  high  degree  of 
confidence  can  be  placed  In  the  reliability  of  the  procedure  for  per¬ 
forming  TCA.  Of  the  disagreements  that  did  occur,  the  vast  majority 
again  arose  when  a  Judge  had  an  erroneous  Impression  of  what  a  device 
did  or  did  not  do. 


3 . 3  Physical  Similarity  Amiris  (PSA) 

The  second  major  analytic  step  in  applying  the  model  consists  of 
Physical  Similarity  analysis.  This  analysis  is  derived  from  several 
traditional  fl del  1 ty- type  measures,  and  deals  in  some  detail  with  the 
similarity  between  physical  characteristics  of  the  training  device  and 
those  of  the  operational  situation.  The  assessment  Is  based  on  the 
physical  similarity  or  fidelity  of  displays  and  controls  in  the  training 
device  relative  to  those  In  the  operational  equipment. 

3.3.1  Data  Requirements.  The  detailed  task  descriptive  data 
developed  during  TCA  for  both  the  training  device  and  the  operational 
equipment  are  used  to  generate  a  list  of  controls  and  displays  relevant 
to  each  subtask.  These  lists  constitute  the  basic  Input  to  PSA.  A 
display  is  defined  as  an  Information  source  or  transmitter,  and  a  contrpj 
a s  an  1  nf ortiwtj on  receiver  which  must  be  physically  opera t ed  on .  Infor¬ 
mation  is  defined  in  the  Information  theoretic  sense  used  by  Fitts  and 
Posner  (196/).  A  control  or  display  is  included  In  the  list,  generated 
tor  each  subtask  if  it  either  transmits  or  receives  the  information  In¬ 
volved  in  performance  of  the  subtask. 

In  Table  3,  the  two  major  tasks  together  with  their  subordinate 
subtasks  are  listed  along  the  left  margin.  Under  each  subtask  the  dis¬ 
plays  (D)  and  controls  (C)  Involved  directly  In  subtask  performance  are 
also  listed.  In  subtask  M,  for  example,  information  Is  transmitted 
by  1)  an  earphone  displaying  the  alert,  element  oi  the  fire  command; 
and  2)  by  the  momentary  on/off  position  of  the  turret  power  toggle 
switch.  The  control  in  this  subtask  consists  of  the  turret-power  toggle 
switch  Itself. 

3.3.2  PSA  Procedure,  for  each  suotask,  a  rating  is  made  on  each 
relevant  control  and  display  which  describes  how  well  It  is  represented 
in  the  training  device.  While  ratings  of  subtasks  lacking  In  coimiunality 
are  not  used  directly,  It  Is  generally  useful  to  make  ratings  for  all 
subtasks.  The  ratings  of  physical  similarity  are  made  along  the  following 
four-point  scale: 
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TABLE  3 

SIMILARITY  ANALYSIS 


TASK  lj_ .  Eire  Main  Gun  (M-32  Sight) 

SuB  task's ; 

1.  Alert 

D|  Farph'ones  (gunner) 

Dp  On/off  pos  of  turret  power 
swl tch 

C,  Turret  power  toggle  switch 

2.  Select  Gun 

0]  Car  phones  (airno) 

0;  On/off  pos  of  Main  Gun  Toggle 
Switch 

Dj  On/off  pos  of  coaxial  machine 
gun  switch 

Cj  Main  gun  toggle  switch 
Cp  Coaxial  machine  gun  switch 

3 .  I ndex  Computer 

[) )  FaniKone's  Cantnb ) 

Dj  Indexing  Window 
D3  Indexing  habile  feedback 
C]  Indexing  handle 

4 .  Monitor 

I)i  FarpKones  (target) 

FH  Uni  t y  Window 

C)'  Microphone  (Identified) 

•  I nl  tlal  Aim 

0 3  Cadillac  control  feedback 
D-)  M-32  Sight  (target) 

O3  Reticle 

C,  Cadillac  elevating-traversing 
control 

C2  Palm  switches 
6.  Track 

Di  fr'IF'Sight  (dynamic  target) 

D?  Reticle 

cf  Cadillac  elevatl ng-traverslng 
control 

tp  Palm  switches 


Slmilarl ty 

Physical  Functional 

it: —  ~rr — 

17-4  B4  M55  SIM  17-4  B4  M55  SIM 


2  2  3  3 
2  2  3  3 


0  0  0  0 


2  3  3 
2  3  3 
2  3  3 


0  0  0  0 

2  2  3  3 

113  3 

113  3 


0  0  0  0 

2  2  3  3 

0  0  0  0 


1  3 
0  3 


0  0  2  2 

2  3  3  3 

13  3  3 

0  3  3  3 


3  3  3  3 
3  3  3  3 


3  3  3  3 
2  2  3  3 
113  3 
113  3 


3  3 
2  2 
3  3 


0  0  3 
2  2  2 
3  3  3 

2  3  .3 
0  3  3 


0  0  2  2 

0  0  3  3 

0  0  3  3 

0  3  3  3 
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TABLE  3  (Cont'd) 
SIMILARITY  ANALYSIS 


7  .  F  i  na  1  A  i  m 
0,  Earphones  (fire) 

[).,  Primary  sight  (target) 

U*j  Reticle 

Ci  Cadillac  elevating-traversing 
control 

C’\  Calm  switches 

«. 

» .  I  i  re 

i).  ( arphone  (up) 
cj  Microphone  (on  the  way) 

C  -  Trigger  switches 


S  Iim  1  Tar  1 


Ph^s  ica  1 


Functional 
“7 T- - 


17-4  04  MS 5  SIM  17-4  B4  MbS  SIM 


0  0  0  0 

2  ?.  2  2 

2  3  3  .3 

13  3  3 

0  3  3  3 


n  0  o  o 

o  n  o  o 

?  3  3  3 


3  3  3  3 
3  2  11 
3  3  3  3 

2  3  3  3 
0  3  3  3 


3  3  3  3 
3  3  3  3 
3  3  3  3 


TASK  It  : . Adjust  lire  Using  HOT  (M-32  SijhtJ 

<l>ubtasks ; 


1  .  Relaj 

t) i  nVe  feedback  rock  -bang)  0 

0-,  M-32  Sight  (target)  2 

O3  Reticle  2 

Ci  Cadillac  elevating-traversing 

control  1 

C;>  Calm  switches  0 


0 

c 

3 

3 

3 


0 

*1 

i 

3 

3 

3 


O 

1 

3 

3 

3 


0  0  0  0 

2  2  ?  ?. 

3  3  3  3 

2  3  3  3 

0  3  3  3 


7.  Sense 

0|  W-7JT  sight,  (burst) 

D;i  M-32  siqht  (tarqet) 

03  M-32  sight  (reticle) 

3.  Apply  HOT 

Oi  32  si ght  (target) 

Do  Reticle 

C‘,  Cadillac  elevating-traversing 
control 

Cj>  Calm  switches 


1111 

2  2  2  2 

2  3  3  3 


2  2  2  ? 

2  3  3  3 

13  3  3 

0  3  3  3 


1111 
2  2  2  2 

3  3  3  3 


2  2  2  2 

3  3  3  3 

2  3  3  3 

0  3  3  3 


4 .  Track 

D]  fCIT" sight  (dynamic  target) 

Do  Reticle 

C]  Cadillac  elevating-traversing 
control 


0  0 
2  3 

1  3 


2  2 
3  3 

3  3 


0  0 
3  3 

2  3 


2 
3 

3  3 


CJ  Cl 


Similar  1  tv 


C2  Palm  swl tches 
5.  Fire 

Oi  Earphones  (up) 

Cj  Microphone  (on  the  way) 

C2  Trigger  switches 

Initial  Interjudqe  agreement 
(3  of  4  agree) 

Consensual  Interjudge  agreement 
(4  of  4  agree) 


Physical 


Func  tlonal 


17-4  B4  MSS  SIM  17-4  B4  MS5  SIM 


0000  3333 
0  0  0  0  3  3  3  3 
2  3  3  3  3  3  3  3 


.66  .76  .97  .86  .79  .86  .71  .75 


.93  .93  .93  .93  1 .0  1 .0  1  .0  1  .0 


Rating  OeJIriitlori 

3  Identical.  The  trainee  would  not  notice  a  difference 

between  the  training  device  control  or  display  and  the 
operational  control  or  display  at  the  time  of  transfer.  Note 
that  they  need  not  be  absolutely  Identical,  but  there  must  be 
no  '‘Jnd"  {just  noticeable  difference)  for  the  trainee.  Include 
for  consideration  the  location,  appearance,  feel,  and  any  other 
ph/s leal  characteristics.  Ignore  the  amount  and  quality  of 
Information  transmitted. 

t  Similar.  There  would  be  a  jnd  for  the  trainee  at  the  time  of 

transfer,  but  he  would  be  able  to  perform  the  task,  There 
might  he  a  decrement  in  performance  at  transfer,  but  any  such 
decrement  would  bo  readily  overcome. 

1  Dissimilar,  There  would  be  a  large  noticeable  difference, 

quite  apparent  to  the  trainee,  at  transfer,  and  a  large 
performance  decrement,  given  that  the  trainee  could  perform 
at  all.  Specific  instruction  and  practice  would  be  required 
on  the  operational  equipment  after  transfer  to  overcome  the 
decrement. 

0  The  control  or  display  is  riot  represented  at  all  in  the  train¬ 

ing  device, 

The  ratings  are  then  entered  In  the  appropriate  cell  of  the  task 
similarity  nv»trfx  (see  Table  ')). 

3.3.3  Results,  The  ratings  of  physical  similarity  for  the  displays 
and  controls  associated  with  each  subtask  are  shown  in  Table  3  for  the 
four  training  devices  under  consideration.  Notice,  as  one  might  expect., 
that  the  training  devices  which  make  use  of  the  actual  operational  equip¬ 
ment  (e.g.,  M55  and  SIMF1RE )  rate  quite  highly  In  terms  of  physical 
similarity.  The  17-4  and  17-B4  devices  tend  to  have  somewhat  lower 
ratings.  This  Is  particularly  true,  for  instance,  in  subtask  1-1 
where  earphones  are  not  used,  and  in  subtask  J»5  where  Cadillac  control 
(jerk)  feedback  is  simply  not  present. 
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Once  the  ratings  for  each  subtask  are  completed  It  Is  possible  to 
generate  a  summary  Index  of  display-control  similarity  at  the  subtask  level 
within  each  device.  First  the  mean  rating  for  the  subtask  is  obtained,  by 
summing  the  ratings  and  dividing  by  the  number  of  displays  and  controls 
rated  for  the  subtask.  For  example,  the  average  rating  for  subtask  1-5  as 
represented  In  the  17-4  (see  Table  3)  Is  obtained  by  summing  the  17-4  simi¬ 
larity  ratings  for  the  1-5  displays  and  controls  (0+2+2+1+0  *  5),  and  dividing 
by  the  number  of  displays  and  controls  rated  In  subtask  !-5(s),  providing 
an  average  rating  of  5/5  •  1.  This  average  rating  will  always  be  a  positive 
number  between  0  and  3i  to  provide  a  similarity  Index  scale  ranging  from  0 
to  1  the  average  rating  Is  divided  by  3.  For  subtask  1*5,  the  17-4  gets  an 
Index  value  of  { 5/ 5 ) / 3  *  .33;  the  same  subt&sk  receives  a  higher  physical 
similarity  score  as  represented  In  SlMflRE,  l.e.,  (l4/5)/3  ■  .93. 

3.3.4  Evaluation.  The  feasibility  of  performing  PSA  was  related  to 
the  availability  of  detailed  task-descriptive  data  on  the  one  hand,  and  on 
the  other  to  the  analyst's  familiarity  with  the  operational  equipment  and 
training  device.  Conducting  PSA  on  the  basis  of  Information  contained  In  a 
TOR  would  be  difficult  unless  data  were  available  from  other  sources  about 
the  displays  and  controls  comprising  the  operational  gear.  These  Inputs 
coupled  with  statements  about  the  degree  of  realism  planned  for  a  device 
might  permit  PSA  to  be  performed. 

The  same  four  Judges  who  performed  TCA  also  were  Involved  in  PSA.  As 
shown  in  Table  3  reasonably  high  Interjudge  agreement  was  obtained  for  the 
four  devices.  In  rating  29  different  controls  and  displays,  Initial  agree¬ 
ment  among  at  least  three  out  of  four  judges  was  obtained  between  661  and  97% 
of  the  time.  Following  resolution  of  definitional  problems  the  agreement 
among  all  four  Judges  reached  931.  Most  disagreements  resulted  from  one  of 
two  sources.  Analysts  were  occasionally  misinformed  about  displays  and  con¬ 
trols  In  either  the  operational  or  training  context.  Additionally,  there 
was  some  difficulty  In  distinguishing  between  scale  values  of  "1"  and  "2H. 

In  the  future  It  may  be  appropriate  to  collapse  these  two  rating  points. 

3.4  Functional  Similarity  Analysis  (FSA) 

The  next  step  In  using  the  model  Is  conducting  a  Functional  Similarity 


27 


analysis  (FSA),  As  Indicated  above,  traditional  f 1 del  1 ty  measures  typically 
concentrate  on  the  representation  of  controls  and  displays,  and  Ignore  the 
behavior  required  of  the  operator  using  the  equipment.  The  activities  of 
the  operator  would  seem  to  be  at  least  as  Important  for  training  as  the 
physical  characteristics  of  the  controls  and  displays  used  to  carry  them 
out.  The  present  analysis  has  been  developed  to  assess  the  adequacy  of 
representation  of  those  activities  In  the  trainlnq  device. 

The  functional  similarity  analysis  examines  the  operator's  behavior 
In  terms  of  the  Information  flow  from  each  display  to  the  operator,  ami 
from  the  operator  to  each  control.  This  examination  Is  made  In  term  of  the 
, (mount,  of  Information  transmitted  (Fitts  and  Posner,  196?)  from  each  display 
to  each  control  (regardless  of  the  actual  operational  mode  of  transmission 
or  reception),  and  the  type  of  Information-processing  activity  performed  by 
the  operator,  Thus,  regardless  of  the  physical  characteristics  of  a  control 
or  display,  the  Issue  Is  whether  the  operator  acts  upon  the  same  amount  of 
information  In  the  some  way  In  both  the  operational  and  training  situations. 

.1.4.1  Data  Requirements.  This  analysis  makes  use  of  the  subtask 
descriptions  and  a  list  of  the  controls  and  displays  In  the  operational 
situation.  These  Inputs  are  used  to  generate  a  flow  diagram  of  each  sub- 
task  which  Indicates  the  type,  amount,  and  direction  of  Information  flow 
for  each  control  and  display.  Cach  situation  In  which  a  display  transmits 
information  to  the  operator  (e.q,,  the  operator  reads  the  display)  Is  de¬ 
fined  as  a  stimulus  function,  and  each  situation  In  which  the  operator 
transmits  information  to  a  control  (e.q.,  operates  It)  is  termed  a  response 
function.  Thus,  the  derived  Input  for  the  Functional  Similarity  analysis 
Is  the  list,  of  Infortnatlon-processinq  functions  indicated  by  the  controls 
and  displays  of  the  operational  situation. 

3.4.2  ISA  Procedure.  In  each  subtask,  the  number  of  bits  of  in- 
t urination  Is  determined  for  each  stimulus  and  response  situation,  by 
estimating  the  number  of  states  which  the  display  or  control  may  assume. 

The  amount  of  Information  In  the  operational  setting  (MQS)  Is  equal  to 
log?  of  the  number  of  states  In  the  stimulus  or  response  functions  under 
consideration.  The  amount  of  Information  in  the  training  setting  (Hts) 
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for  each  of  the  corresponding  functions  Is  estimated  In  the  same  way 
for  each  training  device.  Each  stimulus  and  response  function  1$  then 
rated  according  to  the  following  four-point  scale: 

Rati  mi  Definition 

3  Identical .  *  HQJ. 

2  Similar.  Hqsi  they  are  within  one  logj  unit  of  each 

other. 

1  Dissimilar.  Hts  t  H0Jt  they  are  more  than  one  log^  unit 

apart. 

0  Missing.  HQ5  •  0,  and  •  0. 

In  certain  cases,  ratings  of  2  and  1  will  be  assigned  to  situations 
that  have  been  purposely  made  unequal  by  the  device  designer  In  order  to 
Implement  some  training  technique  (e.g.,  augmented  feedback  or  guidance). 
Such  cases  should  be  noted  for  consideration  In  the  Training  Techniques 
stage  of  the  analysis,  in  other  cases  ratings  of  3  will  be  assigned 
when  the  amount  of  Information  is  the  same  or  nearly  so.  but  when  the 
form  of  the  information  Is  radically  different.  For  example,  In  the 
operational  task  the  operator  might  Index  ammunition  by  pulling  and 
turning  the  Index  handle.  This  handle  could  assume  6  positions! 
therefore.  Indexing  ammunition  Is  a  T og^  6-blt  task.  In  a  training 
device,  the  same  six  alternatives  might  be  present!  however,  ammunition 
might  Instead  be  Indexed  by  pressing  one  of  six  buttons.  The  trainee 
might  process  this  different  Information  in  a  completely  different  way, 
or  use  a  different  strategy  to  deal  with  It.  Such  cases  should  also 
be  noted  for  later  consideration. 

3.4.3  Results.  The  ratings  of  stimulus  and  response  functions 
(denoted  by  the  corresponding  control  or  display)  for  each  subtask  for 
each  of  the  four  devices  under  consideration  are  also  presented  In 
Table  3.  Notice  that  In  some  cases  where  the  physical  similarity  Is 
rated  low,  the  functional  similarity  Is  high,  while  In  other  cases  the 
two  kinds  of  similarity  correspond  to  one  another  quite  well.  The  fact 
that  there  are  differences  suggests  that  this  kind  of  analysis  may  be 
a  valuable  adjunct  to  the  analysis  of  potential  transfer. 
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Once  again  the  separate  ratings  for  each  subtask  may  be  compiled  into 
an  index  which  summarizes  functional  similarity  at  the  subtask  level  with¬ 
in  each  device.  This  is  done  following  the  procedure  already  described 
for  PSA, 

3.4,4  Evaluation.  The  functional  similarity  analysis  was  performed 
independently  by  the  same  four  members  of  the  staff.  Their  experience 
suggested  that  FSA  may  be  a  little  more  difficult  than  the  PSA,  but  not 
prohibitively  so.  It  is  crucial  that  the  rater  understand  the  formal 
concept  of  Information  transmission,  and  that  he  be  able  to  describe 
explicitly  the  information  flow  situation  under  scrutiny.  It  was 
found  that  in  most  cases  the  formal  step  of  drawing  an  Information  flow 
diagram  was  not  necessary,  but  that  on  occasion  it  would  greatly  assist 
raters  who  did  not  have  extensive  experience  with  Information  theory. 

An  understanding  of  types  of  information-processing  activities  (c.q., 
information  reduction,  information  conservation,  etc,)  also  aided 
analysts  In  refining  their  Judgments. 

Thus,  given  the  list  of  control  and  display  functions,  the  Intor- 
mation-flow  diagrams,  and  the  detailed  task  desci i ptlons,  the  functional 
similarity  ratings  are  judged  to  be  feasible. 

The  proportion  of  interjudge  agreement  is  prevented  in  Table  3  for 
each  training  device.  Disagreements  arose  from  three  sources: 

1)  misunderstanding  about  the  situation  being  rated,  ?!)  misunderstanding 
about  appropriate  application  of  information  measurement  to  the 
situation,  and  3)  an  inability  to  discriminate  between  the  values  "2" 
and  "1"  on  the  scale  when  the  amount  of  information  was  relatively 
large  (e.g. ,  in  a  real-world  visual  display).  The  four  sets  of  ratings 
were  resolved  by  consensus,  and  the  resolved  ratings  are  those  presented. 

3 • $  Learning  Deficit  Analysis  (IDA) 

In  order  to  predict  the  effectiveness  of  training  devices,  it  was 
demonstrated  in  the  first  report  in  this  series  (Wheaton  et  al..  1976) 
that  the  analyst  must  not  only  compare  the  content,  and  physical  and 
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functional  characteristics  of  the  operational  situation  and  training 
devices,  but  must  also  consider  the  student  to  be  trained.  It  Is 
Imperative  that  his  capabilities  be  evaluated  relative  to  the  required 
performance  criterion  on  the  operational  task.  Learning  Deficit 
Analysis  (LDA)  Is  designed  to  provide  this  assessment. 

The  learning  deficit  analysis  consists  of  three  procedures  to: 

(1)  assess  the  skills  and  knowledges  In  the  student's  repertory  before 
training,  (2)  determine  the  skills  and  knowledges  which  he  must  possess 
at  the  time  of  transfer  to  the  operational  setting,  and  (3)  estimate  the 
difficulty  (In  terms  of  training  time)  of  training  the  necessary  skills 
and  knowledges.  The  output  of  this  stage  of  the  analysis  Is  a  number 
for  each  subtask  Indicating  the  deficit  possessed  by  the  typical 
trainee,  weighted  by  the  relative  difficulty  (In  terms  of  estimated 
training  time  on  the  operational  equipment)  of  surmounting  that  deficit. 

3 . b . 1  Data  Requirements.  In  order  to  perform  LDA  a  list  of  skills 
and  knowledges  necessary  for  the  adequate  performance  of  each  subtask  Is 
generated.  Beginning  with  the  task  descriptive  data  which  provides 
Information  about  actual  performance  of  the  subtask  on  the  operational 
equipment,  a  sentence  Is  written  which  describes  the  actlvl ty  making  up 
each  subtask.  From  this  statement,  a  list  of  skills  and  knowledges  Is 
developed  for  each  subtask.  The  distinction  between  skills  and  know¬ 
ledges  Is  not  critical  In  this  analysis,  and  Is  made  only  for  the 
convenience  of  the  analyst.  Note  that  the  Information  Input  Into  this 
analysis  (and  correspondingly  its  output)  Is  referenced  only  to  the 
operational  criterion  task  and  to  the  kind  of  trainee  expected.  It 
Is  entirely  Independent  of  any  particular  training  device,  and,  unlike 
any  other  portion  of  the  model,  must  be  performed  only  once  for  each 
subtask  being  considered,  regardless  of  the  number  of  devices  being 
compared . 

3.5.2  Procedure.  The  Learning  Deficit  Analysis  begins  with  the 
application  of  a  rating  scale  to  estimate  the  "amount"  of  each  skill 
or  knowledge  which  the  average  trainee  (of  the  type  selected  for  course 
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enrollment)  could  be  expected  to  have  upon  his  first  exposure  to  the 
training  system  or  device,  It  Is  very  Important  for  the  analyst  to 
keep  In  mind  when  he  makes  his  ratings  that  the  trainee  may  have  had 
some  classroom  training  before  his  exposure  to  a  given  training  device. 

If  what  the  student  learns  in  the  classroom  is  Ignored,  LDA  will  indicate 
a  deficit  larger  than  may  actually  be  the  case,  and  the  model  will  be 
making  a  prediction  regarding  the  effectiveness  of  the  training  device 
without  supporting  classroom  instruction.  While  this  use  of  the  model 
Is  permissible,  the  analyst  will  more  likely  be  Interested  In  the  ef¬ 
fectiveness  of  the  device  in  concert  with  classroom  support,  In  which 
case  deficit  to  be  overcome  by  exposure  to  the  device  will  be  smaller. 

This  latter  type  of  assessment  Is  clearly  fairer  to  the  device,  since 
It  Is  obvious  that  some  things  are  more  effectively  trained  In  the 
classroom.  The  following  Repertory  Scale  (RS),  adapted  from  Demaree 
(1961),  Is  used  to  describe  the  level  of  each  skill  and  knowledge  In 
the  student's  repertory  prior  to  the  start  of  formal  training: 

Rat  Ijhj  Jef  tnltlon 

0  No  experience,  training,  familiarity,  etc:,  with  this  skill 

or  knowledge.  Cannot  perform  a  task  requiring  this  skill  or 
knowledge. 

1  Has  only  a  limited  knowledge  Of  this  subject  or  skill.  Has 
not  actually  used  the  Information  or  skill.  Cannot  be  ex¬ 
pected  to  perform,  Has  had  "orientation"  only. 

2  Has  received  a  complete  briefing  on  the  subject  or  skill. 

Can  use  the  knowledge  or  skill  only  if  assisted  In  every 
step  of  the  operation,  Requires  much  more  training  and 
experience.  Has  received  "familiarization"  training  only. 

3  Understands  the  subject  or  skill  to  be  performed.  Has 
applied  part  of  the  knowledge  or  skill  either  on  the  actual 
job  or  a  trainer.  Has  done  the  Job  enough  times  to  make  sure 
he  can  do  It,  although  perhaps  only  with  close  supervision. 

Needs  mort  practice  under  supervision.  Has  had  "procedural" 
training. 

32 


A 


»■  * 


Rating  Definition 

4  Has  a  complete  understanding  of  the  subject  or  skill.  Can 

do  the  task  completely  and  accurately  without  supervision. 

Has  received  "skill"  training. 

After  the  analyst  has  assessed  the  level  of  skills  and  knowledges 
In  the  trainee's  repertory,  he  proceeds  to  determine  the  required  "amount" 
of  each  skill  or  knowledge  which  the  trainee  must  possess  at  the  close  of 
training  and  time  of  transfer.  The  following  Criterion  Scale  (CS), 
adapted  from  Demaree  (1961)  is  used; 

Rating  Def  inUJon 

0  At  the  end  of  training,  the  trainee  should  have  no  experience 

or  training. 

1  Should  have  a  limited  knowledge  of  the  subject  or  skill.  Has 
not  actually  used  the  information,  Is  not  expected  to  perform 
the  task.  Has  completed  "orientation"  training, 

2  Should  have  received  a  complete  briefing  on  the  subject  or 

task.  Is  able  to  use  the  knowledge  or  skill  only  If  assisted 
In  every  step  of  the  operation.  Requires  much  more  training 
and  experience  to  be  able  to  perform  the  task  Independently. 

Has  had  ‘'familiarization"  training. 

3  Should  have  an  understanding  of  the  subject  or  skill  to  be 
performed.  Has  applied  part  of  the  knowledge  or  skill  on 
the  actual  job  or  a  trainer.  Has  done  the  Job  enough  times 
to  make  sure  he  can  do  It,  although  perhaps  only  with  close 
supervision.  Needs  more  practice  under  supervision.  Has 
had  "procedural"  training. 
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Rating  Definition 

4  Should  have  a  complete  understanding  of  the  subject,  or  be 

highly  skilled.  Is  able  to  perform  the  task  completely, 
accurately,  and  Independently.  Has  had  "skill"  training. 

After  knowledge  and  skills  have  been  rated  on  both  the  repertory 
and  criterion  scales,  the  analyst  calculates  the  learning  deficit  by 
subtracting  the  repertory  rating  ( RS )  from  the  criterion  rating  (CS)  for 
the  knowledge  and  skills  underlying  each  subtask.  Negative  differences 
are  set  equal  to  zero,  since  they  indicate  that  the  trainee  enters 
training  with  more  proficiency  than  Is  necessary,  and  so  has  no  deficit. 

The  difference  scores  thus  range  from  zero  to  four,  the  larger  dif¬ 
ferences  representing  larger  deficits.  The  difference  scores  are 
averaged  within  each  subtask  (collapsing  across  skills  and  knowledges 
in  each  subtask)  to  obtain  a  mean  subtask  deficit. 

The  deficit  score  by  Itself  Ignores  the  fact  that  some  skills  and 
knowledges  are  more  difficult  to  acquire  than  others,  and  thus  some  sub- 
tasks  may  be  more  Important  to  train  than  others.  Presumably,  more 
difficult  subtasks  are  more  critical  for  training,  taklnq  more  time  to 
train,  and  requiring  more  effective  training.  Therefore,  the  next  step 
In  the  LDA  procedure  Is  to  rank  the  subtasks  In  terms  of  estimated  train¬ 
ing  time,  assuming  that  only  the  operational  equipment  would  be  available 
fur  training.  The  analyst  begins  by  seeking  out  the  subtask  whose  estimated 
deficit  would  require  the  least  training  time  on  the  operational  equipment, 
and  assigns  It  a  Dlff Iculty/Crlticallty  rank  of  "1".  The  subtask  re¬ 
quiring  the  next  smallest  amount  of  training  time  for  surmounting  Its 
associated  deficit  Is  assigned  a  rank  of  "2",  and  so  on,  until  all  sub¬ 
tasks  have  been  ranked,  Next,  the  mean  subtask  deficits  are  multiplied 
by  their  corresponding  ranks,  to  obtain  a  weighted  learning  deficit,  score. 
Finally,  each  such  score  Is  divided  by  4  times  the  number  of  subtasks,  to 
provide  an  Index  between  0  and  1  which  reflects  the  size  and  Importance 
of  the  deficit  on  each  subtask  relative  to  the  other  subtasks  being 
analyzed. 
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3.5.3  Results.  The  list  of  skills  and  knowledges  for  each 
subtask  Is  presented  In  Table  4  along  the  left  margin.  These  are 

coded  as  "K1 ,  K2,..."  for  the  knowledges  Involved  In  subtask  1,  "SI,  S2,,.. 
for  the  skills  Involved  In  subtask  1,  and  so  on.  Table  4  presents  the 
difference  between  CS  and  RS  ratings  for  each  skill  and  knowledge,  the 
mean  subtask  difference  scores,  the  subtask  mean  rank  weights,  and  the 
weighted  learning  deficit  scores  as  described  above. 

3.5.4  Evaluation.  The  Learning  Deficit  Analysis  was  performed 
Independently  by  four  senior  members  of  the  project  staff.  Performed 

In  the  manner  described  above,  it  presented  no  significant  difficulties, 
with  the  possible  exception  of  the  generation  of  skill-and-knowledge 
lists.  This  would  not  seem  to  be  a  critical  problem,  since  slightly 
different  lists  would  be  expected  to  lead  to  similar  mean  deficit  scores. 

The  same  staff  members  performed  the  analysis  In  an  alternative  way, 
rank  ordering  indjyjdua 1  skills  and  knowledge,  Instead  of  subtasks,  and 
weighting  the  CS-R5  difference  scores  before  collapsing  within  each 
subtask,  The  scores  obtained  In  this  way  were  highly  correlated  with  the 
weighted  learning  deficit  scores  presented  In  Table  4.  The  correlations 
between  pairs  of  Judges  ranged  from  .90  to  .99,  so  that  this  more  laborious 
procedure  was  felt  to  be  unnecessary. 

Interjudge  differences  In  CS  and  PS  rating  scores  wore  quite  small 
and  unsystematic,  so  ratings  were  averaged  across  the  four  judges.  Inter- 
Judge  agreement  about  subtask  ranks  was  also  quite  high  (correlations 
ranged  from  .85  to  .99),  and  so  ranks  were  averaged  across  judges  in 
Table  4.  Since  similar  high  correlations  on  rating  scores  and  ranks  are 
expected  In  future  appl Icatlons,  the  averaging  procedure  (as  opposed  to 
resolution  by  consensus)  has  been  tentatively  adopted  for  this  analysis. 

3.6  T raining  T ochnlques  Analysis  (TT A ) 

The  next  step  In  predicting  training  device  effectiveness  Is  the 
Training  Techniques  Analysis.  Many  current  devices  have  Incorporated 
special  features  which  are  presumed  to  facilitate  training  beyond  the 
level  possible  on  the  operational  equipment.  Given  the  high  costs 


TABLE  4 


LEARNING  DEFICIT  ANALYSIS 


Subtask 
Difficulty 
Mean  Mean 

CS-RS  CS-RS  Rank . 

Task  I:  Repertory  Item  List  (to  be 

completed  within  S  seconds  for 
stationary  target,  15  sec.  for 
moving) 

X-l  know  procedure 

X-2  Know  control  and  display 

locations 

X-3  Operate  M-32  sight 


Subtask  I:  Alert  3.1/  1.38 

Kl:  X-l  3.50 

K2:  X-2  3.75 

Si.  Operate  control  blindly  2.25 

Subtask  2:  Select  Gun  3.31  2.75 

Kl:  X-l  3.50 

K2:  X-2  3.75 

►  3:  Anno- to- Index  value  trans¬ 
formation  3.75 

SI;  Operate  control  blindly  2.25 

Subtask  3:  Index  Computer  3.75  3.88 

Kl;  X-l  3.75 

K2:  X-2  4.00 

K3;  Anno- to -Index  value  trans¬ 
formation  3.75 

SI.  Operate  control  blindly  3.50 

Subtcsk  4;  Monitor  3.00  4.88 

Kl:  X-l  2.75 

K2 ;  X-2  3.75 

K3 :  Target  descrl ptors-to-target 

transformation  3.00 

SI:  Recognize  targets  visually  2.50 

Subtask  5:  Initial  Aim  3,38  6.25 

Kl;  X-l  3.25 

K2'X-2  375 

K3;  "Center  of  vulnerability"  3.25 

SI:  X-3  3.50 

S2:  Aiming  3.00 

S3:  Detect  transfer  of  control  3.50 


Weighted 
Learning 
Deficit 
Score, _ 


.  14 


.28 


.45 


.46 


.66 


TABLE  4  (Cont’d) 
LEARNING  DEFICIT  ANALYSIS 


Subtask  Weighted 
Difficulty  Learning 


Mean 

Mean 

Def i c 1 t 

CS-RS 

CS-RS 

Rank_  __ 

Score 

Subtask 

6;  Track 

3.50 

7.50 

.82 

Kl : 

X-l 

3.50 

K2: 

X-2 

4.00 

SI : 

X-3 

3.75 

S? : 

Tracking 

2.75 

Subtask 

7 :  Final  Aim 

3.42 

7.25 

.77 

Kl 

X-l 

3.50 

K?: 

X-2 

4.00 

K3: 

"Aim-off" 

3.50 

K4 

"Lead" 

3.00 

SI: 

X-3 

3.50 

S2: 

Aiming 

3.00 

Sub  task 

8 :  Fire 

2.67 

2.1? 

.18 

kl 

X-l 

3.25 

K2: 

X-? 

3.75 

S! 

Estimate  one  second 

1 .00 

Task  11 

Repertory  Item  List  (to  be  com 

pleted  within  15  seconds  for  both 

moving  and  stationary  targets) 

X-l 

know  procedure 

X-2 

know  control  and  display 

locations 

x-; 

Operate  M-32  sight 

Subtask 

1;  Relay 

3.81 

2,50 

.48 

kl 

X-1 

4.00 

K2 

X-2 

4.00 

S! 

X-3 

3.75 

54 

Aiming 

3.50 

Sub  task 

2;  Sense 

3.46 

3.00 

.52 

kl 

X-l 

3.75 

K2 

X-2 

3.75 

k3 

"Sensing"  vs.  "non-sensing" 

3.25 

K4 

"Do  not  announce" 

3.50 

SI 

X-3 

3.50 

S2 

Sensing 

3.00 
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associated  with  many  devices  It  Is  precisely  these  features  which  justify 
using  the  devices  Instead  of  training  on  the  operational  equipment.  This 
stage  of  the  model  attempts  to  assess  the  utility  of  such  features,  and 
to  determine  which  principles  of  acquisition  and  transfer  are  adhered  to» 
and  which  are  violated,  in  the  design  of  the  device.  Thus,  given  common 
content  and  good  representation  of  the  operational  situation,  the  TTA 
attempts  to  answer  the  question:  what  Incremental  training  value  does  a 
specific  device  possess  over  Its  real-world  counterpart? 

The  Training  Techniques  Analysis  relates  the  particular  skills  and 
knowledges  which  must  be  trained  for  each  subtask  to  a  set  of  principles 
and  techniques  which  describe  how  best  to  train  various  kinds  of  content. 
The  techniques  have  been  assembled  from, a  thorough  review  of  the  relevant 
literature  (see  Appendix  A),  and  are  organlted  Into  a  taxonomic  matrix. 
After  identifying  the  appropriate  set  of  techniques,  the  analyst  makes  a 
rating  which  describes  the  extent  to  which  the  device  under  examination 
utilises  the  relevant  principles/techniques  In  order  to  train  a  given  sub¬ 
task. 

3.6.1  Pata  Requirements.  The  task-descriptive  data  and  the  skills- 
and-knowl edges  Information  from  the  Learning  Deficit  Analysis  are  required 
for  the  first  stage  of  the  TTA.  Subtask  descriptions  are  then  assigned 
one  or  more  of  the  following  task-taxonomic  labels  (after  U.S.  Naval 
Training  Device  Center,  1972): 

1.  Recalling  facts  and  principles 

?.,  Recal  1  Ing  procedures 

3.  Non-verbal  Identification 

4.  Non-verbal  detection 

5.  Using  principles,  Interpreting,  Inferring 

6.  Making  decisions 

7.  Continuous  movement 

8.  Verbal  detection  and  identification 

9.  Positioning  and  serial  movement 

10.  Repetitive  movement 
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11.  Written  verbal izatlon 

12.  Oral  verbalization 

13.  Other  verbal ization,  including  signs. 

In  toto,  the  required  input  data  consist  of  the  assigned  taxonomic  labels, 
the  descriptions  of  the  training  device,  any  available  information  on  the 
training  system  within  which  the  device  is  embedded,  and  the  set  of 
training  techniques. 

3.6.2  TTA  Procedure.  The  set  of  training  techniques  1$  organized 
along  two  independent  dimensions.  First,  they  have  been  coded  according 
to  the  taxonomic  category  to  which  they  apply.  Second,  within  each 
taxonomic  category,  they  have  been  further  organized  into  techniques 
relevant  to  stimulus  considerations,  response  considerations,  or  feedback 
considerations.  Thus,  by  referring  to  the  taxonomic  label(s)  which  he 
has  assigned  to  each  subtask,  the  analyst  can  draw  out  those  principles/ 
techniques  which  correspond  to  the  set  of  relevant  behavioral  categories, 
and  sort  them  Into  three  yroups,  stimulus,  response,  and  feedback,  With 
the  operational  task  information  and  the  training  device  and  system 
information  before  him,  he  rates  the  training  device  for  each  relevant 
principle  in  each  of  the  three  categories.  While  performing  the  rating 
operation,  he  should  pay  special  attention  to  any  items  from  previous 
portions  of  the  analysis  which  were  "flagged"  for  attention  at  this 
stage  (e.g.,  see  section  3.3.2).  The  ratings  are  made  from  the  following 
scale; 

Ratify  De_fjnjt  ion 

3  Optimal  Implementation  of  this  technique;  in  complete  accord 

with  this  principle. 

2  Good  Implementation  of  this  technique;  in  excellent  accord 

with  this  principle. 

1  Fair  implementation  of  this  technique;  good  accord  with  this 

principle. 

0  This  principle  or  technique  was  Inapplicable  or  Irrelevant. 

OR 

The  device  neither  Implemented  this  technique  nor  violated 
this  principle. 


Rating  Ocf ini tlon 

•I  Mild  violation  of  this  training  principle;  Implementation  of 

a  mildly  opposing  technique. 

•2  Serious  violation  of  this  principle  or  technique. 

-3  Complete  violation  of  this  principle;  implementation  of  a 

strongly  contraindicated  technique. 

For  each  subtask,  the  lowest  obtained  rating  for  each  of  the 
stimulus,  response,  and  feedback  considerations  is  selected,  and  these 
three  ratings  are  then  averaged  to  obtain  the  training  technique  score 
for  the  subtask.  This  score  is  then  added  to  the  constant  3  (to  delete 
negative  signs),  and  Is  divided  by  6  to  provide  an  index  between  0  and  1 
yielding  the  training  technique  score  of  the  training  device  for  each 
subtask.  Note  that  this  index  Is  very  conservative  since  it  is  quite 
sensitive  to  violations  of  principles/techniques.  This  index  was  consid¬ 
ered  preferable  to  one  based  on  all  the  ratings  since  the  negative  Impact 
ot  violations  of  principles  was  considered  most  critical  In  determining 
overall  device  effectiveness. 

3.6.3  Results.  The  outcome  of  the  TTA  is  shown  in  Table  S.  The 
table  shows  the  taxonomic  categories  assigned  to  each  subtask,  as  well 
as  the  lowest  ratings  of  the  stimulus,  response,  and  feedback  components  of 
each  subtask  across  devices.  Also  shown  are  the  transformed  and  averaged 
indices  computed  across  devices  for  each  subtask. 

The  patterns  of  averaged  Indices  provide  several  kinds  of  informa¬ 
tion.  For  example,  the  lowest  Index  value  (i.e.,  .00)  is  obtained  on 
subtasks  1-6  and  1 1 -4  for  the  17-4  and  I7-B4  devices.  The  lack  of  a 
provision  for  the  training  of  tracking  in  these  two  devices  accounts  for 
this  rather  low  rating.  On  other  subtasks,  however,  these  devices  rom- 
pare  favorably  with  the  M55  or  SIMFIRE  devices  (e.g.,  subtask  1-4). 
Similarly,  the  SIMFIRE  device  receives  a  rating  of  .00  for  subtask  1 1-2 
because  its  design  limits  the  training  which  can  be  provided  for  "sensing" 
the  round.  In  this  case,  violations  of  training  principles  were  noted 
for  the  stimulus,  response,  and  feedback  aspects  of  the  subtask. 
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table  s 

TRAISIfiG  PRINCIPLES/ TE OtJi I QL  ES  ANALYSIS 


and  3  »«re  applicable  to  all  subtasks  of  Tasks  I  and  II  in  addition  to  those 


3.6,4  Evaluation.  The  Training  Techniques  Analysis  was  performed 
In  two  stages:  the  assignment  of  the  taxonomic  labels  to  the  skill  and 
knowledge  components  of  subtasks,  and  the  rating  of  the  relevant 
prtnc iples/technlques.  Four  members  of  the  project  staff  independently 
performed  the  first  stage,  and  found  It  to  be  relatively  straightforward, 
particularly  If  they  were  not  constrained  to  assign  only  one  label  to 
each  skill  or  knowledge.  The  assignment  of  taxonomic  labels  was  reasonably 
consistent.  Most  Interjudge  disagreements  resulted  from  Incomplete 
understanding  of  the  behavioral  categories  denoted  by  the  taxonomic 
labels.  It  was  not  necessary  to  resolve  disagreements  of  this  sort, 
since  the  effect  of  disagreement  here  was  only  to  Increase  the  number  of 
relevant  principles  to  be  noted.  Since  the  labeling  process  was  designed 
only  to  save  the  analyst's  time,  It  (s  felt  that  use  of  multiple  labels 
Is  satisfactory  at  the  model's  present  stage  of  development. 

The  rating  of  the  application  of  principles  for  each  device  was 
conducted  by  two  project  staff  members  working  together,  so  no  direct 
reliability  data  are  available.  Their  Impression  was  that  while  the  assign¬ 
ment  of  ratings  was  time-consuming  and  awkward,  the  consensual  Judgments 
were  fairly  stable.  A  reasonable  alternative  to  the  present  rating  scale 
Is  to  assign  the  zero  point  to  a  given  training  configuration  (e.g.,  the 
operational  cqulpncnt)  and  assign  Training  Technique  ratings  to  devices 
relative  to  this  standard. 

This  aspect  of  the  procedures  for  application  of  the  model  Is 
currently  under  revision.  The  analysis  itself  is  viewed  as  high  In 
potential  for  enhancing  the  predictive  accuracy  of  the  model. 

3 • 7  Summary  Indices 

The  analysis  described  above  (e.g. .  TCA,  PSA,  FSA,  LDA,  and  TTA) 
constitute  a  series  of  analytic  procedures  focused  at  the  subtask  level. 

The  Army,  however,  will  rarely  be  Interested  In  predictions  of  transfer 
based  on  any  one  analysis,  or  for  a  single  subtask.  This  section  develops 
the  procedures  for  collapsing  across  subtasks  to  obtain  the  three  Indices 
Implied  by  the  model:  Transfer  Potential,  Learning  Deficit,  and  Training 
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Techniques.  The  obtained  Indices  for  these  three  components  are  presently 
not  used  In  calculating  overall  training  device  effectiveness,  but  may  In 
fact  be  diagnostic  of  a  device's  particular  deficiencies  or  assets. 

The  problem  of  collapsing  the  Indices  across  subtasks  and  analyses 
is  most  easily  viewed  as  a  problem  related  to  fuzzy  set  theory  In  logic 
and  mathematics.  Any  system  Including  a  training  device,  an  operational 
setting,  and  a  training  objective  can  be  described  In  terms  of  a  set  of 
attributes  or  properties  (e.g.,  communal Ity,  similarity,  etc.),  which, 
when  taken  together,  contribute  to  transfer  of  training.  The  problem  Is 
to  determine  how  to  collapse  the  measures  of  each  attribute  so  that  they 
reflect  the  global  property,  "transfer  of  training".  The  solution  is 
derived  conceptually  In  the  following  sections  (see  Allen,  1974,  for  a 
more  formal  presentation  of  the  mathematics  Involved). 

3.7.1  Transfer  Potential  Index.  Transfer  potential  Is  defined  In 
the  model  as  a  Joint  function  of  communallty  (C)  and  similarity  (S). 

This  transfer  potential  function  Is  characterized  by  the  notion  that  C 
and  S  limit  each  other;  l.e.,  similarity  of  non-conmunal  subtasks  does 
not  contribute  to  transfer  potential,  nor  does  communallty  without 
similarity;  but  when  a  subtask  is  both  in  common  with  the  operational 
situation,  and  possesses  some  similarity,  transfer  potential  exists.  Such 
an  and  relationship  Is  described  mathematically  as  a  multiplicative 
operation.  Thus,  for  any  subtask  1,  the  transfer  potential  of  that  sub¬ 
task  Is  defined  as  times  S^.  Note  that  this  definition  makes  good 
Intuitive  sense  as  well.  When  both  and  S1  are  greater  than  zero, 
transfer  potential  Is  also  greater  then  zero.  If  either  Is  zero, 
transfer  potential  for  that  subtesk  Is  zero. 

The  relationship  between  subtasks  is  somewhat  different.  The 
similarity  In  subtask  1  does  not  contribute  to  transfer  potential  In 
subtask  J,  and  the  same  holds  for  communal Ity.  Transfer  potential  for 
the  two  usks  combined  should  Increase  as  the  transfer  potential  for 
either  subtask  1  or  J  Increases.  Or  relationships  are  described 
mathematically  by  addition.  Thu*,  for  any  two  subtasks  combined, 
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transfer  potential  could  be  defined  as  (C^  X  Sj)  +  (Cj  X  Sj } .  As 
described  In  Sections  3.2  and  3.3,  and  S(  vary  between  0  and  1 .  In 
order  to  scale  this  combined  Index  between  0  and  1,  It  Is  then  appropriate 
to  divide  the  sum  by  2.  In  general,  the  overall  transfer  potential  for  a 
set  of  subtasks  may  be  defined  as  N 

£«MXst> 


1«1 


where  N  is  the  number  of 


N 


subtasks  in  the  set  being  analyzed, 

3./. 2  Learning  Deficit  Index.  In  section  3,5,  an  Index  for  the 
learning  deficit  of  each  subtask  (D)  was  defined  as  the  difference 
between  the  Criterion  Scale  and  Repertory  Scale  ratings,  weighted  by  the 
rank  difficulty  of  the  subtask  and  divided  by  a  constant  to  scale  It 
between  0  and  1.  Learning  deficit  for  a  task  should  Increase  when  two 
subtusks  1  and  j  are  combined,  provided  that  and  Dj  are  greater  than 
zero,  suggesting  that  addition  across  subtasks  is  appropriate.  Therefore, 
the  overall  leurninq  deficit  index  for  a  set  of  subtasks  may  be  defined 
as  H 

L(»i> 

i*i  ’  . 

3.7.3  Training  Technique  Index.  In  section  3.6,  an  Index  (T)  for 
the  contributions  of  training  techniques  In  each  subtask  was  defined  as 
the  mean  of  the  lowest  ratings  for  stimulus,  response,  and  feedback 
principles,  transformed  by  constants  to  scale  it  between  0  and  1.  As 
argued  above,  T  for  subtasks  !  and  j  should  be  independent.  Therefore, 
the  overall  training  technique  Index  may  be  defined  as  N 

l(T1> 

1*1  1  . 
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Each  of  these  Indices  is  necessary  In  order  to  predict  transfer; 
but  none  by  itself  Is  sufficient,  provided  the  others  are  available  (see 
Wheaton  et  al.,  1974a,  on  the  development  of  the  structural  model).  What 
remains,  therefore,  is  to  Indicate  how  one  combines  these  three  indices 


ch  is  capable  of  predicting  transfer,  This 
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4.0  PREDICTION  OF  TRAINING  DEVICE  EFFECTIVENESS 


4 . 1  Introduction 

The  goal  of  the  present  project  Is  to  predict  training  device 
effectiveness.  The  criterion  selected  for  effectiveness  Is  transfer  of 
training  (Wheaton,  et  al . ,  1976),  This  section  presents  the  development 
of  an  equation  whose  parameters  are  the  outcomes  of  the  analyses  discussed 
In  Section  3.0.  It  predicts  values  which  are  components  of  a  traditional 
transfer  of  training  formula  (GagnA,  Foster,  and  Crowley,  1948). 

4 . 2  Empirical  Transfer 

The  formulae  used  to  describe  experimentally  obtained  transfer 
effects  have  been  discussed  In  the  two  previous  reports  In  this  senes, 
so  only  a  brief  review  Is  necessary  here.  One  of  these  formulae  has 
been  tentatively  adopted  for  use  with  the  model.  This  formula  (GagnA, 
et  al.,  1948)  expresses  transfer  (7)  In  terms  of  the  savings  In  time, 
trials,  or  errors  achieved  by  an  experimental  group  with  pretraining, 
relative  to  a  control  group  with  no  pretraining,  to  reach  a  specified 
criterion  on  the  transfer  task.  This  Is  formally  expressed  as 
Tm  where  C  Is  the  number  of  trials  or  errors,  or  the  amount  of  time 
required  by  the  control  group  to  reach  the  criterion,  and  E  Is  the 
trials,  errors,  or  time  required  by  the  experimental  group  to  reach  that 
same  criterion,  after  some  amount  of  pretraining.  For  example,  suppose 
a  control  group  was  trained  In  main  tank  gunnery  on  the  M60A1  tank,  and 
required  26  hours  of  training  to  pass  a  gunnery  proficiency  test.  An 
experimental  group  Is  pretrained  on  a  gunnery  simulator,  and  then  Is 
transferred  to  the  M6GA1  tank,  where  they  require  10  hours  of  additional 
training  to  pass  the  same  gunnery  proficiency  test.  In  this  experiment, 
transfer  would  be  calculated  as  follows:  C"25;  E*1 0;  • jy-  *  .60, 

Indicating  a  savings  of  601  In  practice  time  on  the  operational  equipment. 
Note  that  this  formula  does  not  consider  the  total  amount  of  time  that 
the  experimental  group  spent  on  training,  but  rather  the  time  to  criterion 
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after  transfer,  so  that  the  transfer  value  is  always  specific  to  a 
combination  of  the  kind  and  amount  of  pretraining  the  experimental 
group  received. 

4 . 3  Predicting  T 

In  order  to  predict  T  as  defined  by  Gagni,  et  al .  (1948),  it  is 
necessary  to  predict  the  two  parameters,  C  and  E:  the  time,  trials,  or 
errors  required  by  the  control  and  experimental  groups  respectively  to 
reach  criterion  on  the  transfer  task.  The  control  group  Is  trained  to 
criterion  exclusively  on  the  operational  equipment,  while  the  experimental 
group  is  first  trained  on  a  training  device,  and  then  trained  to  criterion 
on  the  operational  equipment  (i.e.,  the  transfer  task). 

The  control  group  may  be  thought  of  as  a  group  which  Is  trained  on 
a  training  device  having  perfect  commonality  and  similarity  with  the 
transfer  device,  since  they  are,  In  fact,  the  same  device.  In  the  cur¬ 
rent  form  of  the  model,  it  is  also  assumed  that  the  training  techniques 
for  the  control  group  are  optimal.  (The  implications  of  this  assumption 
will  be  discussed  later.)  The  level  of  proficiency  (C)  which  the  control 
group  will  achieve  after  some  arbitrary  amount  of  time  is  based  on  three 
components:  1)  the  content  which  is  trained  relative  to  the  content 
which  is  to  be  tested;  2)  the  difficulty  of  acquiring  the  content  re¬ 
quired  by  the  criterion;  and  3)  the  value  of  the  techniques  employed  to 
train  the  content.  The  content  trained  relative  to  the  content  to  be 
tested  is  estimated  by  the  transfer  potential  portion  of  the  present 
model,  and  is  expressed  for  any  subtask  1  as  C(  X  S^.  The  difficulty 
In  mastering  the  content  required  in  subtask  1  Is  calculated  by  the 
learning  deficit  portion  of  the  model,  and  is  expressed  as  .  Finally, 
training  techniques  employed  are  described  by  the  training  techniques 
portion  of  the  model,  and  their  value  is  expressed  as  T^.  For  any 
particular  subtask,  then,  the  amount  learned  by  or  proportion  of  deficit 
overcome  by  the  group  can  be  estimated  by  Cj  X  Sj  X  X  T^.  The  time, 
trials,  or  errors  to  a  criterion  on  subtaik  1  Is  assumed  to  be  a  linear 
function  of  C<  X  S,  X  X  Tr  Since  it  was  assumed  that  $1#  and  Tj 
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v«uld  be  1  for  all  subtasks  for  the  control  group,  the  estimated  time, 
trials,  or  errors  for  the  total  task,  will  reduce  to  a  linear  function  of 
N 

;  0^.  Thus,  the  model's  estimate  of  C  In  the  Gagn£  et  al.  formula  Is 

1*1 

some  linear  function  of  N 

£  v 

1-1 

The  experimental  group's  score,  E,  can  be  considered  as  resulting 
from  two  components:  1)  the  amount  of  deficit  overcome  on  the  operational 
equlnmcnt,  and  2)  the  amount  of  operational  deficit  overcome  by  exposure 
to  the  training  device.  The  amount  learned  before  transfer  can  be 
derived  as  was  done  above  for  the  control  group,  except  that  Cj,  ,  and 
Tj  do  not  necessarily  equal  1  ,  since  the  training  situation  for  the 
experiment-- 1  group  need  not  be  identical  to  the  operational  setting,  as 
was  the  case  for  the  cont  ol  group.  Therefore,  the  amount  learned  before 
transfer  (l.e.,  on  the  training  device)  Is  estimated  by  N 

*  51  X  Di  X  T 

1*1 

The  amount  that  renv  Ins  to  be  learned  after  transfer  for  the  experiment 

group  Is  then,  "hat  c he  total  deficit  was  minus  what  was  learned  on  the 

training  device:  N  N 

£cixsjXOjXTi‘  The  time,  trials,  or  er- 

i»l  1*1 

rors  to  rMterlon  can  be  estimated  by  a  linear  function  of  this  expression. 
Thus,  the  model's  istlmate  of  E  Is  some  linear  function  of 

n  n 

l  0^  -  ^  X  X  X  Tj .  Finally,  the  value  of  T  can  be  estimated 
1-1  1-1 

by  substitution:  let  estimated  T  be  written  as  T;  let  estimated  C  •  N 


let  estimated  L  -  N  ^ 

Vo,  -  ^  X  s,  xDj  nt; 

1-1  i-1 

then  N  N  N 

-  j;u,  ♦  Jlci  X  Si  X  D1  X  T( 

r  -  i-i  i-i  1-1 _ 

£°i 
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where  C^.  ,  T^,  and  are  calculated  as  prescribed  in  Section  3.0. 

This  formula  reduces  to  a  linear  function  of,  N 

t  ci  X  S1  X  T1  X  D1 


1-1 

n 

This  formula  for  T  has  several  interesting  and  desirable  properties, 
since  derivations  from  it  correspond  to  both  empirical  findings  and 
theoretical  predictions.  First,  as  the  task  communal  1 ty  (C)  Increases, 
predicted  transfer  (In  terms  of  savings)  Increases.  Second,  as  similarity 
(S)  increases,  predicted  transfer  also  Increases,  Third,  as  training 
techniques  (T)  improve,  predicted  transfer  increases.  Further,  all  three 
of  these  effects  are  moderated  cr  weighted  by  the  difficulty  or  deficit 
([))  of  the  relevant  subtaskc. 

To  illustrate  this  last  property,  consider  a  two-subtask  case 
where  subtask  1  is  difficult,  while  subtask  2  is  easy,  further,  suppose 
there  are  two  training  devices  designed  to  teach  the  whole  task.  One 
device  has  high  communal i ty,  similarity,  and  training  techniques  for 
subtask  1,  and  low  communal ity,  similarity,  and  training  techniques  for 
subtask  2,  the  other  device  is  just  the  opposite.  Hypothetical  values 
and  calculations  are  presented  In  Table  6  for  T.  The  estimated  T % 
clearly  favor  the  device  which  is  good  at  training  the  hard  task,  as  would 


i  ; 
}  . 
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II. 


Subtask  1 


.8  (hard) 


Subtask  2 


c<  SH  Ti  Cl  Si  Ti 

111  000 


,3  (easy)  000 


1 


1.1 


EC|  s,  1,  D,  • 

1  *1  *1  •  ( .8)  ♦  0* 
0-0-( .3)  -  .8 


ICi  Si  T,  Oi  ■ 

0-0*0  •{. 8)  +  1  • 
1*1  •(  .3)  ■  .3 
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Intuitively  be  expected.  This  weighting  effect  operates  as  well  In  less 
clear-cut  cases. 

One  caveat  needs  to  be  sounded.  It  was  assumed  that  the  relation¬ 
ship  between  the  amount  to  be  learned  (or  remaining  to  be  learned)  and 
time,  trials,  or  errors  to  a  criterion  Is  a  nearly  linear  function,  Such 
relationships  In  fact  are  known  not  to  be,  in  general,  linear.  The 
relationship  between  time  and  performance  has  generally  been  found  to  be 
a  mono ton 1c  increasing,  negatively  accelerated  curve  for  acquisition 
situations  (e.g.,  control  group,  or  experimental  group  before  or  after 
transfer;  c.f.  Hull,  1951).  Time  before  transfer  and  performance  after 
transfer  may  have  art  even  more  complex  relationship  (c.f.  Mandler,  1962). 
Nevertheless,  It  seems  reasonable  at  the  present  time  to  characterize  these 
relationships  as  approximately  linear  In  situations  of  Interest  to  Army 
trainers.  Clearly,  an  Important  topic  for  further  research  Is  the 
relationship  between  amount  of  practice  and  performance  In  transfer/ 
acquisition  situations  represented  In  the  Army. 

As  mentioned  earlier,  there  are  also  problems  with  assuming  all 
T^s  »  1  for  the  control  group.  This  suggests  that  the  operational  equip¬ 
ment  always  assures  the  best  training  techniques,  an  assumption  which 
contradicts  the  hopes  of  device  designers  who  Incorporate  training 
features  in  devices  (such  as  "problem  freeze",  augmented  feedback,  etc.). 

It  might  just  as  well  have  been  assumed  that  all  control  T(s  •  .5,  a 
neutral  value,  but  this  assumption  would  have  been  just  as  unfounded. 

What  Is  clearly  required  Is  a  training  technique  analysis  which  references 
the  operational  equipment.  This  does  not  seem  too  difficult  to  construct, 
and  Is  an  area  for  early  Improvement  of  the  model. 

Predicted  7$  have  been  generated  for  the  devices  considered  In 
Section  3,0  above,  and  these  are  presented  in  Table  7.  <s  are 
presented  for  each  device  with  respect  to  the  two  major  tasks  selected 
for  analysis,,  Further  Ts  have  been  estimated  for  each  device  with  and 
without  supoortlng  classroom  Instruction  for  both  potential  and  actual 
subtask  cornnunallty.  The  table  also  presents  estimated  T  s  for  the 
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TABLE  7 


TRAINING  DEVICE  EFFECTIVENESS  PREDICTIONS:  ESTIMATED  " 


Training  Utilization  Device 


Task  1  -  Fire  Main  Gun 

17-4 

17-B4 

M56 

SIMFIRE 

Potential 

.  1 349 

.2142 

.3721 

.3721 

Potential  with  supporting  classroom 
instruction 

.1175 

.2007 

.3711 

.3711 

Actual 

.1349 

.2142 

.3427 

.3427 

Actual  with  supporting  classroom 
instruction 

.1175 

.2007 

.3490 

.3490 

TaiK  II  •  Adjust  Fire  Using  BOT 

Potential 

.1285 

.2333 

.3884 

.3219 

Potential  with  supporting  classroom 
Instruction 

.1175 

.1962 

.3912 

.3213 

Actual 

.1081 

.1885 

.3362 

.26  9  7 

Actual  with  supporting  classroom 
Instruction 

.0981 

.1536 

.3417 

.2735 

Combined  (1  and  II) 

Potential 

.1323 

.2220 

.3788 

.3516 

Potential  with  supporting  classroom 
instruction 

.1175 

.1984 

.3812 

.3470 

Actual 

.1240 

.2037 

.3400 

.3129 

Actual  with  supporting  classroom 
instruction 

.1077 

.1771 

.3453 

.3111 
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combined  tasks.  Note,  however,  that  this  combined  value  Is  not  simply  an 
average  of  the  f  s  for  the  two  subtasks;  It  Is  weighted  by  the  number  of 
subtasks,  t'ti!  1-arnlng  deficit,  and  individual  subtask  T  $.  It  is  also 
true  that  ’  •  ;iot  directly  comparable  between  tasks;  primarily,  this 
is  due  to  the  ranking  procedure  of  the  Learning  Deficit  analysis.  While 
two  subtasks  (one  In  each  task)  might  have  the  same  "1"  difficulty/ 
criticality  rank,  they  might  not  be  equally  easy  to  learn.  Thus,  while 
a  higher  value  of  T  'ls  Interpreted  as  better  transfer  potential,  this 
estimate  is  relative  to  the  other  devices  under  consideration  for  a  given 
task, 

Table  7  reveals  some  other  Interesting  Information  as  well.  First, 
there  is  a  large  and  differential  impact  on  predicted  effectiveness  for 
the  different  utilizations.  For  example,  there  is  no  difference  between 
actual  and  potential  transfer  predicted  for  the  17-4  and  17-B4,  while 
there  Is  a  noticeable  decrement  for  the  M55  and  S I MF IRE.  Similarly,  the 
effectiveness  of  the  17-4  and  17-B4  devices  Is  reduced  when  supporting 
classroom  instruction  Is  provided;  the  predicted  effectiveness  generally 
improves  with  classroom  support  for  the  M55  and  SIMFIRL.  There  are  clear 
differences  between  the  two  tasks  studied:  while  the  M55  and  S IMF  IRE  are 
equivalent  for  Task  I,  the  MSS  is  clearly  superior  for  Task  II. 

While  it  Is  Important  to  explore  the  determinants  of  these  results, 
any  such  speculations  would  be  without  empirical  Justification.  Clearly, 
experimental  evidence  as  to  the  relative  efficacy  of  these  devices  for 
these  hypothetical  utilization  situations  is  needed. 


5.0  DISCUSSION 


5 . 1  Composition  of  the  Model 

As  presently  structured,  the  training-effectiveness  model  provides  for 
relatively  comprehensive  treatment  of  factors  directly  related  to  a  training 
device  which  are  known  to  Impact  upon  transfer  of  training.  There  are  other 
factors  which  should  certainly  be  considered  for  Inclusion  In  future 
versions  of  the  model.  These  typically  Involve  features  of  the  training 
system  external  to  the  device  Itself;  two  of  the  more  Important  of  these 
are  the  amount  of  training/practice  given  and  Instructor-student  acceptance 
of  the  device.  Work  on  the  potential  impact  of  these  kinds  of  training- 
system  variables  will  continue. 

As  far  as  device-related  variables  go,  one  additional  construct  might 
be  added  Inmedlatcly  to  the  present  model:  a  third  similarity  analysis  which 
assesses  the  degree  to  which  various  adverse  conditions,  expected  to  Impact 
on  task  performance,  are  simulated  In  the  device.  The  data  required  for  such 
an  Environmental  Fidelity  Analysis  (EFA)  could  be  obtained  by  Interviewing 
experienced  operators  as  to  the  special  or  adverse  conditions  which  occur 
from  time  to  time  and  affect  task  performance  (e.g.,  extreme  temperature, 
reduced  visibility,  etc.).  Building  upon  procedures  suggested  by  Chenzoff 
and  Tolley  (196b),  It  should  be  possible  to  estimate  how  severely  each 
condition  degrades  performance,  how  likely  It  Is  to  occur,  and  what  specific 
subtasks  are  Impacted  upon.  From  descriptions  of  the  training  device,  esti¬ 
mates  could  then  be  made  regarding  its  capability  for  simulating  each 
adverse  condition. 

Insofar  as  possible,  an  attempt  has  been  made  to  Incorporate  the  rele¬ 
vant  thinking  and  constructs  of  other  investigators  Into  the  model.  To 
the  extent  that  we  have  been  successful  in  doing  so,  the  model  represents 
a  coalescing  of  Ideas  about  the  nature  of  transfer  and  the  factors  which 
Influence  It. 

5.2  Data  Requirements 

The  Initial  applications  of  the  model  described  In  earlier  sections  of 
the  report  were  undertaken  for  two  reasons.  First,  such  exercises  afforded 
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an  opportunity  to  Identify  and  resolve  any  procedural  difficulties  which 
might  arise.  Second,  and  perhaps  even  more  importantly,  they  provided 
for  detorrrl nation  of  the  model's  Information  requirements  and  for  assessment 
of  the  feasibility  of  satisfying  those  requirements.  At  issue  were  the 
Kinds  of  data  needed  and  the  anticipated  quality  of  that  data  as  a  function 
of  the  type  of  device  being  evaluated  and  Its  stage  of  development. 

S.?.1  Kinds  of  Input.  The  basic  data  requirements,  as  Identified 
during  conduct  of  the  primary  analyses  (I.e.,  TCA,  PSA,  FSA,  IDA,  and  TTA) 
are  threefold,  first,  a  detailed  statement  of  the  training  objectl ve(s)  Is 
mandatory  for  the  device  under  Investigation.  Second,  detailed  task-analytic 
data  of  the  operational  task  and  the  training  situation  are  required.  Third, 
estimates  are  needed  of  the  capabilities  and  existing  knowledge  of  the 
trainee  population  who  will  practice  on  the  device. 

The  Army  has  developed  detailed  procedures  for  the  specification  of 
training  objectives.  As  discussed  In  CON  Reg.  350-100-1  (1972)  a  training 
objective  Is  to  contain:  The  action  which  the  trainee  must  be  able  to  per¬ 
form,  the  conditions  under  which  he  is  expected  to  perform,  and  the  stan¬ 
dards  of  performance  he  must  reach.  The  action  element  determines  which 
specific  tasks  are  to  be  evaluated  In  TCA,  PSA,  and  FSA.  It  also  contributes 
to  IDA.  The  conditions  element  would  contribute  to  an  Environmental  Similarity 
Analysis  and  to  IDA.  Finally,  the  standards  element  is  vital  for  accurate 
determination  of  the  LDA. 

Provision  has  also  been  made  for  the  generation  of  detailed  task- 
analytic  Information  as  described  In  CON  Reg.  350-100-1  (197?)  and  CON 
Pam.  350-11  (1973).  Particularly  relevant  are  the  data  generated  during 
training  analysis  which,  when  cast  into  Job-Task-Data-Card  format,  Indicates 
the  task,  subtask,  job  task  conditions,  job  task  standards,  skills  and 
knowledge,  and  attitudes.  These  categories  of  data  are  directly  relevant 
to  most  of  the  analyses  required  by  the  model.  It  should  be  noted,  however, 
that  these  Inputs  are  descriptive  of  the  operational  or  transfer  situation. 
Analogous  data  are  required  for  description  of  the  training  situation.  At 
present  there  appears  to  be  no  formal  provision  for  the  generation  of  such 
Information.  This  fact,  as  will  be  discussed  below,  has  Implications  for 
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the  kinds  of  devices  and  stages  of  development  which  can  be  addressed  by 
the  predictive  model . 

The  last  kind  of  Input  required  by  the  model  Is  not  available  In 
any  formal  lied  sense,  but  can  be  obtained  fairly  readily.  In  order  to 
specify  parameter  values  during  IDA,  estimates  are  needed  of  the  capabilities 
of  the  trainees  who  are  to  use  a  given  device.  It  seems  that  the  kind  of 
input  which  is  required  can  be  obtained  from  a  general  consideration  of 
trainees'  backgrounds  (l.e.,  the  kind  and  amount  of  prior  military  training 
which  they  have  received). 

6.2,2  Quality  of  Input.  The  key  Issue  In  practical  applications  of 
the  current  model  Is  the  feasibility  of  acquiring  the  Input  data  which 
are  needed.  As  discussed  above,  several  different  kinds  of  input  are 
necessary  at  rather  detailed  levels  of  description.  Of  concern,  therefore, 

Is  the  assumption  that  the  quality  of  this  Input  (and,  consequently,  of 
the  output)  may  vary  as  a  function  of  other  factors.  Two  of  these,  considered 
during  the  applications  reported  above,  are:  1)  the  system  or  nonsystem 
orientation  of  the  device  under  evaluation;  and  2)  the  stage  during  design 
and  development  when  the  evaluation  Is  attempted. 

The  Army  defines  a  training  device  as  any  three-dimensional  object 
developed,  fabricated,  or  procured  specifically  for  Improving  the  learning 
process.  These  devices  are  classified  as  system  or  nonsystem.  System 
devices  are  those  designed  for  use  with  one  system  or  Item  of  equipment 
(e.g,,  for  the  TOW  missile,  the  M6QA1  tank,  the  M 1 6  rifle).  Nonsystem 
devices  are  designed  to  support  general  military  training  and/or  for  use 
with  more  than  one  system  or  Item  of  equipment  (e.g.,  a  burst-on-target 
trainer,  or  a  main  gun  trainer).  While  this  distinction  may  be  valid  for 
other  reasons,  It  is  not  Important  within  the  context  of  the  current  model 
since  It  treats  all  devices  as  though  they  were  system  devices .  This  view 
stems  naturally  from  the  way  In  which  the  evaluation  Is  conducted.  The 
basic  Input  always  consists  of  the  training  objective,  which  either  Indicates 
directly,  or  certainly  Implies,  the  specific  operational  context  to  be  con¬ 
sidered.  The  model's  concern  with  transfer  of  training  as  the  measure 
of  effectiveness  forces  consideration  of  the  specific  operational  system 


or  equipment  for  which  training  Is  being  given.  For  Instance,  when  the 
effectiveness  of  the  17-1  Burst-on-Target  trainer  Is  to  be  estimated,  the 
estimate  must  be  made  for  some  designated  system.  This  Is  In  no  way  meant 
to  Imply  that  the  17-4  would  necessarily  be  judged  equally  effective  when 
applied  to,  for  Instance,  the  M60A1  tank,  the  M16  rifle,  or  the  105»m 
howitzer  systems.  While  the  type  of  device  does  not  Influence  the  quality 
of  Inputs  to  the  model,  the  manner  In  which  the  device  1$  represented  cer¬ 
tainly  does. 

The  model  provides  an  estimate  of  training  effectiveness  based  upon 
an  analysis  of  both  the  operational  context/equipment  and  the  training 
situation/device.  The  amount  and  quality  of  the  Information  which  Is 
available  to  describe  either  of  these  settings  is  a  function  of  their  stage 
of  development.  This  point  quickly  becomes  apparent  when  one  considers 
the  operational  equipment  and  the  training  device  at  both  conceptual  and 
prototyplc  levels  of  development. 

Four  situations  arise  In  which  one  might  went  to  apply  the  model  to 
forecast  device  effectiveness.  In  the  first  Instance,  assume  that  both 
the  operational  equipment  and  the  device  are  at  least  at  the  prototype 
stage  of  development .  In  this  case  the  quality  of  the  Information  which 
could  be  generated  would  be  at  a  maximum.  However,  the  utility  of  the  re¬ 
sultant  forecast  would  be  diminished  by  the  extent  to  which  large  costs 
had  already  been  Incurred  In  producing  the  device  prototype.  In  the 
second  case,  assume  that  the  equipment  was  In  prototype  form  and  that  the 
device  was  represented  by  a  Training  Device  Requirement  (TDR).  Here, 
clearly,  a  forecast  of  effectiveness  would  have  great  utility.  Unfortunately, 
however,  the  TDR's  as  currently  written  do  not  provide  the  Information 
required  for  description  of  the  device  (AR  71-7,  1973).  The  third  and 
fourth  cases,  In  which  either  the  equipment  or  both  the  equipment  and  device 
were  described  in  preliminary  document  form,  would  prove  similarly  unmanage¬ 
able.  No  single  document  could  be  found  which  contelned  all  of  the  Information 
required  for  analysis,  nor  would  the  available  Information  be  of  sufficient 
quality. 
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This  situation  does  not  preclude  early  assessment  of  a  device.  It 
does  suggest,  however,  that  current  TDR's  would  need  to  be  reformatted  If 
they  were  to  serve  as  the  single  source  for  all  necessary  Input  data  to 
the  model.  While  this  may  represent  a  significant  problem,  It  Is  felt 
that  much  of  the  needed  Information  may  be  available  from  different  sources. 
Aggregation  of  this  data  should  be  feasible. 

5.3  Analytic  Procedures 

The  procedures  employed  In  the  several  analyses  demanded  by  the  model 
appear  to  be  reasonably  sound.  Most  of  them  represent  a  compromise  over 

choice  of  appropriate  level  of  analysis.  By  working  at  what  seems  to  be  an 

Intermediate  level  of  detail  It  is  possible  to  Introduce  a  reasonable 
degree  of  rigor  and  precision  while  avoiding  the  extremely  time-consuming 
effort  which  has  plagued  other  approaches. 

The  most  difficult  and  lengthy  processing  Involved  occurs,  in  fact, 
during  the  generation  of  Input  data.  The  need  for  task-analytic  Infor¬ 
mation  Is  Inescapable,  and  one  must  continue  to  go  through  this  complicated 

procedure.  Within  the  model  Itself  the  Training  Techniques  analysis,  as 
Indicated  in  Section  3.0,  Is  rather  laborious.  In  the  future,  attempts  must 
be  made  to  conduct  this  analysis  at  the  subtask  level  rather  than  at  the 
level  of  Individual  knowledge  and  skills.  Similarly,  ratings  should  be  made 
relative  to  the  operational  equipment  which  the  device  Is  designed  to  replace 
for  training  purposes. 

5 . 4  Validity  of  the  Model 

Development  of  the  predictive  model  is  obviously  Intended  to  replace 
the  costly  approach  of  empirical  evaluation  of  proposed  devices  and  to 
permit  evaluations  earlier  In  the  life-cycle  of  the  device.  Before  reliance 
can  be  placed  on  the  model,  however,  Its  own  effectiveness  must  be 
determined.  This  means  that  Its  predictions  must  be  checked  against 
empirically  obtained  results.  This  process  of  verification  represents 
validation  of  the  model. 

As  Indicated  In  the  Second  Interim  Report  (Wheaton,  et  el.,  1974), 
''validation"  of  the  model  actually  refers  to  two  kinds  of  validity:  a) 
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predictive  validity,  and  b)  construct  validity,  Predictive  validity 
addresses  the  question  of  whether  the  model's  output  is  useful  in  predicting 
the  relative  effectiveness  of  different  devices.  Construct  validity  refers 
to  the  degree  to  which  the  dimensions  of  the  model  hypothesized  to  influence 
transfer  actually  measure  or  represent  what  they  purport  to  measure,  In 
essence,  construct  validity  addresses  the  theoretical  structure  of  the  model, 
its  internal  aspects  rather  than  its  output.  It  Is  clear,  therefore,  that 
predictive  and  construct  validation  serve  two  different  purposes. 

Predictive  validity  is  assessed  by  the  covariance  between  the  model’s 
output  and  some  concurrent  criterion  measure  of  transfer  of  training.  In 
perhaps  the  most  basic  case  the  model  can  make  a  prediction  about  the  relative 
transfer  of  training  arising  from  two  training  programs—one  which  employs 
one  device  and  another  In  which  instruction  is  provided  by  exposure  to  a 
different  device.  In  this  Instance,  the  model  would  predict  the  relative 
effectiveness  of  alternative  devices. 

Construct  validity  Is  demonstrated  by  determining  the  relationship 
between  specific  predictor  variables  (e.g.,  various  components  of  the  model) 
and  transfer  measures,  In  this  kind  of  study  Interest  lies  In  determining 
the  effects  which  parametric  variations  of  Independent  predictor  variables 
have  on  the  criter1a-«1n  this  case  transfer-of-trainlng  measures.  One  can 
develop  confidence  In  the  value  of  any  construct  and  In  the  construct 
validity  of  a  set  of  measures  when  It  Is  found  that  devices  which  result  In 
good  training  and  devices  which  train  poorly  differ  in  some  ways,  and  that 
these  differences  are  In  accord  with  theoretical  (model)  predictions. 

In  the  present  project  predictive  validity  Is  of  more  Immediate  concern 
than  Is  construct  validity.  In  essence,  an  attempt  has  already  been  made 
to  build  construct  validity  into  the  model.  Constructs  were  Included  only 
after  an  extensive  review  of  the  literature,  and  only  where  powerful  effects 
on  transfer  had  already  been  d*nonstrated.  The  remaining  construct  validity 
questions  and  experimental  studies  must  await  the  results  from  the  predictive 
validation  efforts.  Consequently,  the  field  validation  effort  which  Is 
being  planned  will  focus  on  whether  or  not  the  model's  predictions  can  be 
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corroborated  by  obtained  data.  If  predictive  validity  Is  demonstrated  then  1 

attention  can  return  to  the  construct  validity  of  various  dimensions  com¬ 
prising  the  model . 
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APPKNDIX  A  TRAINING  TVOINIQUKLI  AND  PRINCIPLES 


An  moot  toned  In  the  first  report  in  thin  series,  the  literature 
dealing  with  transfer  of  traininq  is  voluminous.  Despite  the  enorinouri 
accumulation  of  empirical  informat ion,  the  data  are  generally  still  in 
a  form  which  in  Loth  unsystematic  and  difficult  to  use.  Very  few  at¬ 
tempts  have  been  mode  to  organ! no  thin  Literturo  into  a  usable  body  of 
Knowledge.  Major  attempts  are  represented  in  the  work  of  Willis  and 
Peterson  1  anti  Michel if  Ho  tw  maximally  effective,  a  traininq  psychologist 
must  bo  able  to  organize  and  synthesize  the  relevant  literature  in  order 
to  extrapolate  to  a  specific  training  situation. 

Hu*  approach  employed  hy  Willis  and  Peterson  and  modified  by  Micheli 
was  to  survey  the  literature  in  order  to  derive  a  wot  of  common  generali¬ 
zations  or  principles  about  learning  and  transfer.  A  particular  training 
situation  couUI  then  i*>  evaluated  against  these  principles  to  estimate 
the  eft  icacy  of  a  specific  training  configuration.  Hie  basic  organizational 
technique  was  to  create  a  t>r inciplos-hy-taok  matrix  and  to  generate  training 
guidelines  tot  each  of  the  resultant  cells.  These  writers  employed  slightly 
vi i f r'*;rc*nt  catwjor ical  taxonomies  for  class i f icatior.  of  tasknj  despite  these 
d 1 1 1 oroneer. ,  t*>th  systems  ,ier  vod  primarily  as  aids  to  organizing  training 
gu  lik*  lines, 

Fur  ixirpor.es  of  tJie  nronent  model  evaluation  exerciser, ,  tills  name 
mat r i x  approach  war.  enploycd  for  the  Training  Hobniqucs  arntlysis.  Mach 
t  >iw  in  the  mat  r  i  x  represented  a  uubVask  anti  each  column  represent  ml  a 
iqxviftc  training  principle' technique,  Ho  manner  in  which  a  device 
actually  provided  for  training  of  each  sub  task  was  then  writ  ranted  with 
the  pi  incipler,/ techniques.  TV,  aid  in  this  process,  each  axis  of  the  matrix 
was  nystomattzod.  With  reajwn't  to  the  r.ubtaitk  axis,  tan*  categorizations 
wen*  used,  tatted  oo  Michel l 'ii  n~catoqory  in:ht*mo .  Kui  the  principle/ 
techniques  axis,  a  rough  distinction  artotig  stimulon,  response,  and  f»Nx1l>ar,'k 
pr  UK'iplon  war.  used. 

Table  A- l  presents  the  dint r ibuticMi  of  principles  across  task  cate¬ 
gories.  In  the  present  evaluation#  the  analyntn  une.1  this  table  ptimaiily 
as  a  guideline |  in  cases  where  a  task  categorization  war.  douiitful,  the 
principles  tor  all  conceivable  cat  rxjor  i  zat  ions  were  ncanned.  Table  A-? 
presents  the  set  of  principles  employed  in  the  present,  study.  Hi  esc 
pr  irx'ipler.  were  developed  from  three  major  sources:  Willis  and  Peterson, 


Willis,  M.  P.  k  Peterson,  R.  0.  Deriving  training  device  impl icat.ions. 
from  theory  principles.  Volume  Ii  Guidelines  for  traininq  device 
design,  development,  and  use.  U.S.  Naval  Training  Device  Center, 

Port.  Washington,  New  York,  1961,  AD  264  364. 

In  n.S.  Naval  Training  Device  Center.  Staff  study  on  cost  and  training 
effectiveness  of  proposed  training  systems,  TART  Report  1,  F.S.  Naval 
Training  Device  Center,  Orlando,  Florida,  1972. 


Michel i,  and  the  online?  review  of  the  wplrieal  literature.  Fach  of 
the  principles  adopted  from  either  Willis  and  Peterson  or  Mieheli  was 
evaluated  against  the  literatures  empirical  and  theoretical  support 
for  each  principle/technique  was  then  ' ’rated." 

It  should  be  re-emphas i xcd  that  the  behavioral  taxonomy  and  principle 
groupings  served  primarily  to  aid  the  analysis  in  locating  relevant, 
principles.  Other  organisational  and  reference  systems  are  currently 
being  developed,  an  well  as  more  precise  statements  of  the  principles 
thcmaeWee, 
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P  *  1  N  C  !  P  L  l  S 


TASK  CATEGORY 

STIMULUS 

RESPONSE 

FEEDBACK 

R«?Ci>1  1  -  F«t»  A 

Pr  t  (VC  i  pi  «B 

1,2, 3,4, 5 

1,2,3,26,31 

<.2.27,29,30 

Reel  11  •  Procedures 

2. 4, 5, 6, 7, 8, 9 

1.2,3,26,29,31 

1.2,3.29,30 

Non-VerUil 

tdentlf  lutlon 

10, 1 1 . 12, 36 

4.6,26 

1  ,4,5,27,29.30 

Non-Vert>»l  Detect  ton 

13,14 

6 

6,29,30 

Using  Principles, 
Interpreting,  { 

Inferring 

IS, 16, 36 

! 

MM8,19#Z9,)0 

7,27,29,30 

Hiking  Dec  ISions 

17, 18, 19, 20,  .71 , 36 

>.7, 8.9, 17, 18, 19, 29, Jo! 

i  8.29.30 

Contlmjocs  Movement  j 

22,73.24,25,15,36 

10,11,17.20,22,23,24, 
27.28,31  , 

9.10.16,17,19,22,23,24. 
25,26,77 ,26 ,79 , 30 

Vertii  1  Detect  1  i»n  ind  ! 
Uentll  Kittnn  ; 

2.4,6.26 

M2  i 

I 

11,29,30 

Posit  loMng  i  Serl.sl  ! 
HovcncffiJ.  ! 

! 

27, 28.36, 36 

| 

1,13, 71. 23, 74.75, 2?,  ■ 

28,31 

9, 12.15,  If.  17,18, 19, 70. 

5  21  ,72 ,23 ,74  ,27,28 , 

1  79,30 

Repet i t Iwe  Movement 

1 

29. JO, 31. IS  1 

JB, 20. 23, 74, 75, 27, 28, 

31 

1  9. 16, 17,18, 19, 20. 71.??. 

1  23, 24, 26.27, 28, 79. 30, 

Written  Verbil  1/itlon 

,12,33 

1,9 

13.29,30 

1 

Cfil  VurrMt  fr*t1on  . 

29 

14,15,16,2? 

14,79,30 

i 

Other  Vert*  1  fjr.s  t  ion , 
Including  Signs 

. . . . . j 

3.1,34 

1.9,18 

14,29,30 

Table  A-.. 

TRAINING  PRINCIPLES/ TECHNIQUES 


Stimulus  Considerations 

1.  Organise  training  around  Intrinsic  cut  components  (hey  words,  formull,  or  key  let.  jrs) 
within  the  fact  or  principle.  Use  these  cut  components  as  mediators  to  trigger  recall 
of  complete  facts  or  principles. 

Empirical  •  good 
Theoretical  -  indifferent 

2.  Use  mnemonics  (associating  recall  of  facts  or  principles  with  Imagery,  rhyme,  rhythm, 
etc. ). 

Empirical  ■■  good 
Theoretical  -  fair 

3.  Prevent  decay  of  recall  by  (ncreesinq  the  meaningful  ness  of  the  material  to  be  learned 
by  providing  organtmian  to  the  related  facts  or  principles. 

Empirical  -  good 
Theoretical  -  good 

4.  Prevent  decay  of  recall  by  overlearning  the  original  material, 

Empirl  cal  -  yood 

Theoretical  •  moderate 

5.  Prevent  decay  of  recall  by  providing  periodic  refresher  training. 

Empirical  -  fair 

Theoretical  •  excellent 

6.  Use  Mental  rehearsal  of  sequential  steps.  If  readily  codable  In  symbolic  form 
(mediators) . 

Empirical  ■  IMifferent 
Theoretical  -  good 

7.  Haxlmtte  control  of  cues  to  ensure  that  the  trainee  Is  formlnq  the  proper  associations, 
l.e..  responding  appropriately  to  the  correct  cues. 

Emplr  leal  -  good 
Theoretical  -  excel  lent 

8.  In  training  for  recall  of  lengthy  or  difficult  procedures,  develop  redundant  cup 
response  patterns  (via  primary  and  suppl  ementary  stimuli)  to  trigger  the  sequentially 
next  correct  response. 

Empirical  -  fair 
Theoretical  -  good 

9.  Use  programmed  demonstration  of  procedures,  up  to  but  not  beyond  ability  of  student  to 
understand  procedures. 

Empirical  -  good 
Theoretical  -  good 

IQ.  Stlmjll  used  In  training  should  be  nearly  Identical  to  ,)ob  stimuli  unless  this 
fidelity  Increases  problem  difficulty  In  the  Initial  phase  of  training  to  an  un¬ 
acceptable  level . 

Empirical  -  qood 
Theoretical  -  good 
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II.  Vary  ratio  of  relevant  a.d  Irrelevant  (transient)  stimuli  according  to  requirements 
of  various  stages  of  training.  Maximize  relevant  cues  and  minimize  Irrelevant  rues 
In  early  stages  of  training  use  a  realistic  mix  of  relevant  and  Irrelevant  cues  In 
final  stages  of  training. 

Empirical  -  questionable 
theoretical  -  good 

1?.  Emphasize  cuts  which  elicit  mediating  responses,  e.g,,  “self-instructions",  “popula¬ 
tion  stereotypes"  and  “natural  associations". 

Empirical  -  good 
Theoretical  •  good 

13.  Transfer  Increases  as  the  difference  between  reference  end  generalization  stimulus 
decreases. 

Empirical  •  excellent 
Theoretical  -  excellent 

14,  Decrease  ilgnal-to-noise  ratio  at  student  achieves  success  at  a  given  difficulty  level. 
Empirical  •  good 

Theoretical  -  excellent 

15,  Emphasize  the  logical  relationships  which  exist  between  the  general  principle  and  the 
specific  application.  The  unique  or  special  features  of  each  application  should  bn 
minimized  whltr  the  coerwn  relationships  to  the  general  principle  should  be  emphasize). 
Empirical  •  moderate 

Theoretical  •  good 

16.  Stimulus  redundancy  -  apply  principle  in  a  large  number  of  practice  situations,  while 
varying  the  stimulus  context  of  repetitions. 

Empirical  •  fair 
Theoretical  -  good 

1/.  Trainees  must  have  access  to  potentially  relevant  data.  In  final  stage  of  tralninq, 
data  should  be  limited  to  that  expected  In  real  world  situations. 

Empirical  -  fair 
Theoretical  -  good 

18.  Guiding  -  early  in  training  present  logical  Implications  of  alternative  choices. 
Empirical  •  fair 

Theoretical  -  good 

19.  Mediators  -  acquisition  and  use  of  mediators  such  as  stereotypes  or  self -instructions, 
facilitate  the  identification  of  response  alternatives  and  the  probability  of  success 
of  each  alternative. 

Empirical  -  fair 
Theoretical  -  good 

20.  St IxxjIus  load  -  toward  the  end  of  training,  present  trainee  with  a  realistic  data 
processing  load  (realistic  number  of  significant  signets  plus  reellstlc  noise  In 
real  time). 

Empirical  -  fair 
Theoretical  -  good 

?1.  Stimulus  generalization  -  vary  the  stlwlus  context  of  repetitions. 

Empirical  •  fair 
Theoretical  -  ewderate 
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22.  Insure  that  the  appropriate  stimulus  cues  are  available  to  the  trainee  continually 
during  the  performance  of  the  task. 

Empirical  *  good 
Theoretical  •  Moderate 

2).  Emphasis  on  prediction  of  future  states  (thinking  ahead). 

Empirical  •  fair 
Theoretical  *  fair 

24.  Expose  trainee  to  a  wide  range  of  task  difficulty. 

Empirical  •  moderate 
Theoretical  -  good 


25.  In  continuous  control  tasks,  high  fidelity  (s  often  required  In  (1)  stimulus  presenta¬ 
tion,  (2)  operator  response  characteristics,  and  (3)  dynamic  system  behavior,  the 
evolving  display-control  relationship. 

Empirical  -  moderate 
Theoretical  -  good 

26.  Contiguity  •  the  symbol  and  referent  should  be  presented  In  close  temporal  contiguity. 
Empirical  -  good 

Theoretical  •  excellent 

27.  Cue  development  •  emphasize  the  development  and  use  of  Interna)  cues,  such  at 
mediators  or  kinesthetic  cues, 

Imp  Irlcal  •  moderate 
Iheurelkal  -  guod 

28.  In  training  for  lengthy  serial  movements,  provision  should  be  made  for  programing 
demonstrjt  tons  of  the  lengthy  serial  or  sequential  performance  according  to  the  amount 
of  demonstration  which  can  be  understood  by  the  trainee.  Continuing  a  demonstration 
beyond  the  "saturation  point"  mill  result  In  the  association  of  responses  with  In¬ 
correct  cues. 

Empirical  -  fair 
Theoretical  -  good 

2*.  tarly  training  -  use  models  of  correct  performance  as  a  basis  for  trainee  to  perceive 
critical  cues  of  good  form,  use  models  of  component  parts  of  task. 

Empirical  •  moderate 
Theoretical  -  good 

30.  Cue  discrimination  •  perceive  difference  between  correct  and  Incorrect  form. 

Empirical  -  good 
Theoretical  -  excel  lent 

3).  Later  stages  of  training  -  tht  kinesthetic  cues  dominate  (cues  based  on  "muscle  fetl"). 
Empirical  •  moderate 
Theoretical  •  questionable 

32.  Performance  aids  •  especially  In  early  phase  of  training  use  a  performance  aid  or 
model,  such  as  Instructions,  checklists  or  standard  exempItt/fonmaU  to  aid  In 
perceiving  need  for  and  composing  of  messages. 

Empirical  ■  Indifferent 
Theoretical  •  fair 


T*bU‘  A-..  (conc'd) 


33.  *!n*the-head"  mediator*  -  In  later  stages  of  instruction  roly  on  “In-the-heed* 
Instruction*,  models.  *tc,.  to  aid  In  perceiving  need  for  <nd  composing  required 

MttlOtt. 

Empirical  •  Indifferent 
Theoretic*!  •  fair 

34.  Static  end  dynamic  models  -  (n  eerly  phese  of  training  use  node!*,  such  «s  still  end 
moving  grepMc  d  1  splays  (video  tepe  recordings)  to  establish  the  characteristics  or 
criterion  performance. 

Implrlce!  •  fair 
Theoretic*)  ■  fa Ir 

35.  Pre- training  methods  need  to  teke  cere  not  to  make  the  5  dependent  upon  the  fxt/e  cues 
provided  In  the  eerly  stages  of  training  end  thus  to  hinder  the  changeover  to  more 
direct  relations  between  Input  and  output  at  a  later  stage. 

tapirlcal  •  good 
Thaoratlcal  ■  excellent 

36.  With  very  complex  tasks.  Instruction  In  principles  yields  better  results  than  laying 
down  a  detailed  drill,  while  with  simpler  tasks  the  drill  Is  at  least  equally  ef¬ 
fective. 

tmplrlcal  -  good 
Theoretical  -  good 
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Response  Considerations 

1.  Make  an  overt  response  indicating  the  recall  of  facts  and  principles,  enabling 
MMaurmant.  (Add  appropriate  phrase  for  each  behavioral  category.) 

Empirical  •  excellent 

Theoretical  *  aacallant 

2.  Response  Generali  tat  ion  •  make  Job  performance  type  reiponm  ( 1 .  high  fidelity 
responses  later  In  training). 

Empirical  •  questionable 
Theoretical  -  questionable 

1.  Guide  or  prompt  response,  especially  in  the  acquisition  phase  of  training. 

Empirical  -  good 
Theoretical  •  good 

4.  To-be-learned  response  should  occur  as  toon  as  relevant  cuts  art  parceivtd  (con¬ 
tiguous  occurrences  of  cues  and  response). 

tep fries  I  •  good 
Theoretical  •  excellent 

5.  The  strength  of  a  given  response  typically  increases  as  a  function  of  practice. 
Empirical  -  excellent 

Theoretical  -  excellent 

ft.  To  enable  reinforcement  of  performance,  the  student,  upon  detecting  a  signal,  should 
respond  so  that  what  is  detected  end  time  of  detection  can  be  recorded. 

Impirlcel  •  excellent 
Theoretical  -  excellent 

7.  Performance  differences  which  are  due  to  individual  differences  In  ability  tend  to  be 
magnified  as  a  function  of  Increasing  task  difficulty.  Identical  performance  among 
given  trainees  is  nut  necessarily  Indicative  of  Identical  learning,  hence  the  need 
for  e  better  measure  of  the  extent  to  which  trainees  are  profiting  from  the  training 
situation. 

Empirical  -  good 
Theoretical  -  excellent 

8.  Apply  decision  making  In  a  large  number  of  practice  situations  while  varying  the 
stiaailus  context  of  repetitions. 

Empirical  •  fair 
Theoretical  -  good 

9.  Stress  •  when  trainee  will  be  required  to  perform  under  stress,  use  overlearning  of 
skill  to  mlnlml/e  effects  of  competing  responses. 

Empirical  -  fair 
Theoretical  -  good 

10.  Repetition:  highly  skilled  performance  requires  extensive  practice. 

Empir  leal  -  excellent 

Theoretical  •  excellent 

11.  Nike  an  objective  measurement  of  the  frequency  and  type  of  errors  -  changes  In  total 
error  pattern  •  throughout  the  course  of  training.  Measurement  should  be  bated  on 
specific  behavioral  objectives. 

Empirical  •  good 
Theoretical  -  fair 
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1?.  Symbol  -  referent  associations  { pa  1  rings)  are  especially  amenable  to  "In-the-head" 
practice. 

Empirical  •  good 
Theoretical  •  excellent 

13.  Emphasize  extensive  actor  response  repetition  or  practice  In  order  to  (1)  strengthen 
Individual  or  component  ttepi  of  the  movement  series,  end  (2)  Integrate  these  steps 
Into  a  smooth  sequence. 

Empirical  »  excellent 
Theoretical  •  good 

14.  Emphasize  overt  respondlngs  In  a  social  context.  Practice  to  strengthen  correct 
responses. 

Empirical  -  Indifferent 
Theoretical  •  hunch 

i 

15.  Record  responses  In  context.  Since  the  response  Is  typically  complex  Involving  subtle 
relations  a«wng  components,  the  technique  of  measurement  should  provide  for  recording 
the  total  complex  In  a  manner  that  permits  analysts  of  such  subtle  relationships. 
Empirical  *  Indifferent 

Theoretical  •  excel lant 

16.  When  skill  Is  not  regularly  used,  prevent  decay  of  recall  by  providing  periodic 
refresher  training. 

Empirical  •  fair 
Theoretical  >  excellent 

17.  The  usefulness  (for  ‘lateral*  transfer)  of  any  learned  capability  will  be  Increased  If 
tt  Is  practiced  in  as  wide  a  variety  of  situations  as  possible. 

Empirical  •  good 
Theoretical  •  excellent 

18.  Ensure  that  relevant  subordinate  capabilities  have  been  thoroughly  learned  before 
calllno  on  vertical  (e.g.,  Inclusion)  transfer  to  aid  the  learning  of  "advsneed" 
capabilities. 

fep  trlcal  -  good 
Theoretical  •  questionable 

19.  Vertical  transfer  is  enhanced  by  the  variety  of  previous  knowledge. 

Empirical  •  fair 

Theoretical  -  good 

20.  Where  the  whole  task  Is  a  closely  coordinated  activity  such  as  aiming  a  rifle  or 
simulated  flying  of  an  aircraft,  it  Is  better  to  tackle  the  task  as  a  whole.  Any 
attempt  to  divide  it  up  tends  to  destroy  the  proper  coordination  of  action  and 
subordination  of  Individual  actions  to  the  requirements  of  the  whole,  and  thu»  out¬ 
weighs  eny  advantage  there  might  be  In  mastering  different  portions  of  the  task 
separately. 

Empirical  •  good 
Theoretical  •  fair 

21.  Where  the  task  Involves  a  series  of  component  actions  which  hive  to  be  performed  In  the 
correct  order  but  each  Is  largely  Independent  of  the  others,  there  seem  to  be 
advantages  In  practicing  the  different  components  separately. 

Empirical  •  fair 
Theoretical  •  good 
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22.  Continuous  practice  facilitates  mastery  of  complex.  meaningful  material  and  the 
establishment  of  coordinated  rhythmic  activity  (within  limits  of  fatigue), 

Empirical  -  good 

Theoretical  -  fair 

23.  Continuous  practice  seems  to  be  preferred  by  older  trainees. 

Empirical  -  moderate 

Theoretical  -  Indifferent 

24.  Spaced  practice  Is  more  efficient  than  continuous  If  only  the  actual  duration  of 
the  sessions  Is  counted  and  the  time  between  sessions  Is  lonored.  When  the  time 
between  sessions  Is  Included,  continuous  practice  Is  usually  more  efficient, 

Empirical  •  good 

Theoretical  •  Indifferent 

25.  Very  brief  pauses  between  practice  sessions  should  be  as  effective  is  longer  ones. 
Empirical  •  fair 

Theoretical  •  good 

26.  "Mental  practice"  In  which  the  &  performs  a  task  In  the  Imagination,  can  often  be 
substituted  for  a  substantial  amount  of  practice  Involving  full  performance  wl th 
little  If  any  loss  of  effectiveness. 

Empirical  •  fair 
Theoretical  •  good 

V.  Relatively  little  learning  occurs  If  Ss  am  passive  spectators  or  even  passive 
performers,  but  that  they  Rust  be  Involved  In  active  decisions  and  choices  about 
what  they  are  doing,  and  It  Is  these  thet  they  will  retain  whether  they  are 
right  or  wrong. 

Empirical  -  excellent 
Theoretical  •  excellent 

26.  If  two  or  more  tasks  have  to  be  learned,  It  Is  most  beneficial  to  begin  with  the 
one  which  elicits  the  greatest  care  and  effort  towards  the  ettalnnent  of  a  high 
standard  of  performance.  However,  if  S  was  not  allowed  to  continue  to 
practice  the  more  difficult  task  until  a  point  of  reasonable  mastery,  he  would  be 
left  with  an  Inadequate  comprehension  of  the  tesk,  and  transfer  to  a  simpler  task 
might  be  confused  and  less  satisfactory  than  If  he  had  tackled  the  easier  tesk  first, 
Implrlcal  •  good 
Theoretlcel  •  moderate 

29.  The  wore  sub-tasks  there  ere  In  the  overall  task,  and  the  more  they  Interact  with  one 
enother,  the  more  opportunity  there  will  be  for  Improvement,  end  therefore  the  longer 
Improvement  will  continue. 

Empirical  *  good 
Theoretical  •  excellent 

30.  Transfer  of  skill  from  one  task  to  another  will  depend  not  to  much  upon  the  extent 
to  which  methods  possible  for  one  ire  applied  to  the  other,  but  the  extent  to  which 
methods  which  have  Wen"  selected  for  the  one  ere  epplled  to  the  other. 

Empirical  -  Indifferent 
Theoretlcel  -  excellent 

31.  Effectiveness  of  spaclna  practice  depends  on  whet  (s  don#  during  the  timet  between 
practice  periods:  (a)  If  they  are  spent  In  rehearsal  of  the  material,  learning  will 
benefit,  unless  the  task  is  fatiguing  In  which  cate  continued  practice  may  depress 
subsequent  performance,  (b)  If  time  between  practice  periods  are  spent  on  another 
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task,  learning  or  lattr  recall  of  the  first  task  may  be  Impaired,  the  degree  of 
Impairment  depending  on  the  degree  of  similarity  between  the  two  tasks. 
Empirical  •  good 
Theoretical  *  good 


Tmblu  A-i'  (c out. 'il ) 


ElfdjMctS  Considerations 


1.  Schedule  KOR  (knowledge  of  results)  toon  after  response  for  maxima  reinforcement. 
Error  Identification  function  of  KQR  H  significant. 

Empirical  -  excellent 
Theoretical  -  excellent 

2.  Especially  early  In  training,  us*  KOR  after  the  response  to  each  step,  for  maximum 
reinforcement.  Error  Identification  function  of  KOR  It  significant.  In  later 
stages  of  training.  step  feedback  Is  not  to  critical, 

Empirical  -  excellent  (but  mostly  rat*) 

Thaoratkal  *  excellent 

3.  At  training  progresses,  gradually  Increase  the  delay  In  presenting  KOR  (present  KOR 
In  Increments  of  2  steps,  then  3  steps,  etc.)  until  the  tchedulei  of  KOR  approximates 
the  operational  setting, 

Empirical  -  good 
Theoretical  -  excellent 

4.  | mediate  reinforcement  (0.5  second  delay)  for  nonverbal  Identification. 

Empirical  •  no  data  found 

Theoretical  •  hunch 

5.  KOR  -  automatic  system  performance  feedback  (e.g.,  If  target  Is  Identified  from 
partial  cues,  present  the  target  with  a  full  tet  of  Cues  after  Idenllf katlcn; 
automate  if  possible), 

Empirical  -  Indifferent 
Theoretical  *  good 

6.  feedback  omission  schedule  prograevned  according  to  stage  of  training;  high  feedback 
during  initial  stages,  decreased  to  equivalent  to  operational  setting  or  lower. 
Emplrkel  -  excellent 

Theoretical  -  excellent 

7.  Schedule  KOR  soon  after  response  for  maximum  reinforcement.  KOR  should  deal  with 
both  process  and  solution. 

Empirical  -  Indifferent 
Theoretical  -  excellent 

8.  Early  In  training,  evaluate  each  alternative  solutkn  as  It  Is  Identified,  and  when 
«  final  choice  among  alternatives  Is  mad  a,  evaluate  the  overall  choice. 

Empirical  -  Indifferent 

Theoretical  -  hunch 

9.  5  Imping  -  reinforcement  should  be  contingent  upon  characteristic*  of  trainee's 
response  so  that  by  a  process  of  'successive  approx Imatlons ' ,  the  final  desired 
proficiency  Is  produced. 

Empirical  -  excellent 
Theoretical  •  excellent 

10.  Continuous  KOR  -  because  of  the  dynamic  neture  of  the  problem,  the  trainer  should  at 
times  be  presented  with  an  on-going  evaluation  of  his  performance. 

KpTfkal  •  Indifferent 
Theoretical  -  good 
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H.  Incorrect  as  well  <i  correct  sjmibol -referent  pairing  can  be  strengthened  by  self- 
inltlatad  "in-the-haad"  practice. 

Empirical  •  moderate 
Theoretical  -  excellent 

1?.  Extensive  response  repetition  (overlearning)  by  the  trainee  to  take  advantage  of  the 
built' in  feedback  properties  of  theta  types  of  tasks.  Simple  repetitive  movements  may 
be  "automatically"  reinforcing  (Kinesthetic  feedback). 

Empirical  -  excellent 
Theoretical  •  good 

13.  Provide  record  of  trainee's  overt  response  to  enable  evaluation  of  trainee  performance 
(1.9.,  must  know  what  Ss  are  doing  In  ordtr  to  provide  KOR). 

Empirical  -  excel  lent 
Theoretical  *  excel  lent 

14.  Analyte  oral  verbal  nation  recordings  to  evaluate  trainer/team  performance  and 
provide  KOR. 

Empirical  -  txcol  lent 
Theoretical  *  excellent 

15.  Performance  improvement  in  acquisition  depends  on  knowledge  of  results  (KOR).  The 
rate  or  improvmnent  depends  upon  the  precision  of  KOR  (within  limits  of  meaningful- 
nesa), 

Empirical  •  excellent 
Theoretical  -  fair 

16.  Delay  of  KOR  has  little  or  no  effect  on  acquisition  (for  simple  motor  movements  only). 
Empirical  -  excellent 

Theoretical  •  fair 

17.  Increasing  the  poit-KOR  Interval  up  to  a  point  will  Improve  performance  level  In 
acquisition  (spacing  during  practice).  The  "point"  is  when  recall  of  movement  or 
KOR  is  affected. 

Empirical  •  excellent 
Theoretical  -  weak 

IB.  The  type  of  activity  in  the  KOR  delay  or  post-KOR  delay  interval  does  not  influence 
acquisition  (provided  the  intervening  activity  it  not  of  the  same  type). 

Empirical  -  excellent 
Theoretical  •  weak 

19.  Withdrawal  of  KOR  produces  deterioration  of  performance  when  level  of  training  is 
low  or  moderate. 

Empirical  *  excel  lent 
Theoretical  -  excellent 

20.  When  KOR  H  delayed  in  acquisition,  and  S  engages  In  deliberate  verbal  or  motor 
activity  during  the  delay  Interval,  the  effect  of  KOR  withdrawal  it  poorer  performance 
than  when  S  rests. 

Empirical  ‘  -  excellent 
Theoreticel  -  fair 

21.  When  KOR  Is  delayed  in  acquisition,  and  S  rests  during  the  delay  interval,  the  ef¬ 
fect  on  performance  when  KOR  Is  withdrawn  It  no  different  than  wnen  imnedlate  KOR 
It  used. 

Empirical  -  good 
Theoretical  •  fair 
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22.  Activity  In  the  po\t-K'Jtt  delay  Interval  during  acquisition  worsens  performance  when 
K.0K  Is  withdrawn 

Inplrldl  -  excellent 
Theoretical  -  good 

23.  After  a  relatively  Urge  amount  of  training,  learning  can  continue  when  ICOR  Is  with¬ 
drawn. 

Empirical  -  good 
Theoretical  -  fair 

,  If  errors  could  he  prevented  in  the  first  few  trials  (e.g,,  guidance),  mastery  of  the 

test  should  hr  very  much  quicker. 

Empiric al  -  good 
T heurct  k  a  *  ••  e  or  1  lent 

21'.  Go  1  daiH  t>  during  training  Is  beneficial  when  tracking  movements  have  to  be  made  with 
an  1nro.rpat  t  bio  control-display  relationship. 

1  np  I  r  1  c  a  I  •  fair 

Theorntl.  al  -  fair 

2b.  Guidance  duos  not  aid  simple  repetitive  movements,  but  aids  learning  complex  courses. 
Imp  tr  leal  •  gum! 

Theoretical  -  good 

17.  The  runner  c:  vcmvevlng  ION  Is  Important:  (a)  L  f  f  ec  1 1  veness  Is  greatest  when  the 
Information  Is  clearly  and  limply  related  to  the  action  performed.  Any  distortion 
or  rgu  iv-!.  \!  I'm  In  1  fir  information  fe>1  bail  to  the  f,  will  reduce  Us  ef  foctl  vetwss , 
|bj  l.iyluly  kill  *>r  .  .o.plrx  information  ray  be  partly  Ignored  oi  may  contuse  the  S. 

(c)  T  hi>  i  nt  nr m  turn  nlvrn  should  Indicate  the  discrepancy  he  tween  what  Is  required 
and  w»iat  his  tern  achieved  rather  than  merely  ylve  a  reminder  of  requirements  or  some 
hr\iad  measiir»  nf  achievement. 

Ifrplrkal  ■  rvi.fi  lent 
Thesiretk.il  -  excellent 

jii.  Per forman-.  r  1v  best  -amtalned  when  the  conditions  are  such  as  '.o  wnphaslfe  the  need 
for  S  to  observe  Urn  feel  of  his  actions  in  order  to  relate  then  to  their  results, 

Empirical  -  good 
The  or  r  t  leal  -  e»cel  lent 

29.  A  S  must  have  sivr  cues  to  the  results  of  Ms  actions  if  he  Is  to  perform  accurately 
at  all,  and  training  procedures  will  be  effective  In  so  far  as  they  help  him  to 
observe  and  use  si.i  h  cues  as  are  Inherent  In  the  task  for  which  he  Is  being  trained. 
They  will  fall  In  so  far  as  they  pro- lot*  him  with  extra  cues  on  which  ho  comes  to 
rely  hut  which  arc  not  available  when  he  changes  from  training  to  the  actual  Job. 
Empirical  excellent 

Theoretical  -  excellent 

30.  KOft  acts  as  an  Incentive, 
tr.plrlcal  •  excellent 
Theoretical  -  good 


